Skip to content

Using Linux Environment Modules

On RCC systems, most software packages are made available via Linux Environment Modules. These modules allow us to maintain an enormous software library without users having to worry about details such as paths to different software versions or libraries; modules will set or unset the right paths and environment variables for you.

Each module contains the information needed to configure the shell and environment for a specific application. For example, the Intel module sets the environment to enable access to the Intel Compiler libraries and executables. The GNU module does the same for the GCC equivalents. Additionally, you can specify a specific version of an environment. For example, at the time of writing this document, our systems support three Intel compiler versions (v15, v16, and v21).

This page describes general module use, and creating your own custom modules. Our documentation pages for pre-installed software include instructions for loading specific modules when they are needed.

Note

RCC uses lmod, which supports hierarchal modules. Some centers use the environment-modules package, which ships with most Linux distributions.

Note

Some modules are automatically loaded upon login. These include the python and matlab modules. This doesn't preclude you from loading alternative versions of these packages.

Using environment modules#

RCC provides many pre-configured modules for loading software packages, libraries, and compilers. You can see a list of these be running the module avail command, which produces output similar to the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
$ module avail

----------------------------------------------------------------------- /opt/modulefiles/core ------------------------------------------------------------------------
   R/3.5.2                anaconda/3.8.3 (D)    gaussian09            intel/15            matlab/2020a           sagemath/9.4          totalview/2018.2.6
   R/4.0.0                ansysfluent           gaussian09nbo7        intel/16            matlab/2021b  (L,D)    spark                 totalview/2019.2.12
   R/4.1.0         (D)    autodock              gaussian16            intel/21     (D)    pgi/21.3               stata/13mp            totalview/2021.3.9
   R/4.2.0                clang/14.0.0          gembs                 matlab/2013b        proxy                  stata/15       (D)    visit/2.13.3
   agisoft                cuda/10.1             gnu/4.8.5             matlab/2015b        python/2               stata/16mp            visit/3.0.0         (D)
   anaconda/2.7.15        cuda/11.1      (D)    gnu/8.3.1      (D)    matlab/2017a        python/3      (L,D)    stata/16              visit/3.2.1
   anaconda/3.7.3         fastqc                gnu/11.2.1            matlab/2018b        python/3.10.4          totalview/8.13 (D)

--------------------------------------------------------------- /usr/share/lmod/lmod/modulefiles/Core ----------------------------------------------------------------
   lmod    settarg

The output shows you the top-level modules that are available to load. For a full list of modules across all hierarchies, including modules that require other modules to be loaded first, run module spider.

Loading modules#

To load a module, run module load MODULENAME.

For example, if you need to use the Intel C compiler command, icc, you must first load the intel module. If you attempt to run icc before you load the module, you will see an error:

1
2
$ icc --version
-bash: icc: command not found

First load the module, and then you can run the command:

1
2
3
4
$ module load intel
$ icc --version
icc (ICC) 16.0.2 20160204
Copyright (C) 1985-2012 Intel Corporation.  All rights reserved.

Hierarchical modules#

Some module environments depend on other modules to be loaded first. For example, OpenMPI implementations depend on which compiler your code uses: GNU, Intel, PGI, or LLVM/Clang.

For example, you need to use the GNU implementation of OpenMPI. Without a compiler loaded, there are no OpenMPI implementations available. So, you first load the GNU module:

1
$ module load gnu

Once this is done, the modules that depend on GNU will be available to you:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
$ module avail

---------------------------------------------------------------- /opt/modulefiles/compilers/gnu/8.3.1 ----------------------------------------------------------------
   ants/2.3.5    mvapich2/2.3.5 (D)    mvapich2/2.3.7    netcdf/4.7.0    niftyreg/1.3.9    nvhpc/22.3    openmpi/2.1.0    openmpi/4.1.0 (D)    openmpi/4.1.3

----------------------------------------------------------------------- /opt/modulefiles/core ------------------------------------------------------------------------
   R/3.5.2                anaconda/3.8.3 (D)    gaussian09              intel/15            matlab/2020a           sagemath/9.4          totalview/2018.2.6
   R/4.0.0                ansysfluent           gaussian09nbo7          intel/16            matlab/2021b  (D)      spark                 totalview/2019.2.12
   R/4.1.0         (D)    autodock              gaussian16              intel/21     (D)    pgi/21.3               stata/13mp            totalview/2021.3.9
   R/4.2.0                clang/14.0.0          gembs                   matlab/2013b        proxy                  stata/15       (D)    visit/2.13.3
   agisoft                cuda/10.1             gnu/4.8.5               matlab/2015b        python/2               stata/16mp            visit/3.0.0         (D)
   anaconda/2.7.15        cuda/11.1      (D)    gnu/8.3.1      (L,D)    matlab/2017a        python/3      (L,D)    stata/16              visit/3.2.1
   anaconda/3.7.3         fastqc                gnu/11.2.1              matlab/2018b        python/3.10.4          totalview/8.13 (D)

--------------------------------------------------------------- /usr/share/lmod/lmod/modulefiles/Core ----------------------------------------------------------------
   lmod    settarg

You will now see that additional modules are available, including multiple versions of OpenMPI.

Info

Note the annotations beside the modules in the output:

  • (D) stands for default if no version is specified
  • (L) stands for loaded

Now you can load OpenMPI into your path, and you'll see that software that depends on OpenMPI will now become available:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
$ module load gnu openmpi
$ module avail

----------------------------------------------------------- /opt/modulefiles/MPI/gnu/8.3.1/openmpi/4.1.0 ------------------------------------------------------------
   gulp    lsdyna    nektar    netcdf/4.7.0 (D)    openfoam    orca/5.0.1    orca/5.0.3 (D)    turbomole/7.4

---------------------------------------------------------------- /opt/modulefiles/compilers/gnu/8.3.1 ----------------------------------------------------------------
   ants/2.3.5    mvapich2/2.3.5 (D)    mvapich2/2.3.7    netcdf/4.7.0    niftyreg/1.3.9    nvhpc/22.3    openmpi/2.1.0    openmpi/4.1.0 (L,D)    openmpi/4.1.3

----------------------------------------------------------------------- /opt/modulefiles/core ------------------------------------------------------------------------
   R/3.5.2                anaconda/3.8.3 (D)    gaussian09              intel/15            matlab/2020a           sagemath/9.4          totalview/2018.2.6
   R/4.0.0                ansysfluent           gaussian09nbo7          intel/16            matlab/2021b  (L,D)    spark                 totalview/2019.2.12
   R/4.1.0         (D)    autodock              gaussian16              intel/21     (D)    pgi/21.3               stata/13mp            totalview/2021.3.9
   R/4.2.0                clang/14.0.0          gembs                   matlab/2013b        proxy                  stata/15       (D)    visit/2.13.3
   agisoft                cuda/10.1             gnu/4.8.5               matlab/2015b        python/2               stata/16mp            visit/3.0.0         (D)
   anaconda/2.7.15        cuda/11.1      (D)    gnu/8.3.1      (L,D)    matlab/2017a        python/3      (L,D)    stata/16              visit/3.2.1
   anaconda/3.7.3         fastqc                gnu/11.2.1              matlab/2018b        python/3.10.4          totalview/8.13 (D)

--------------------------------------------------------------- /usr/share/lmod/lmod/modulefiles/Core ----------------------------------------------------------------
   lmod    settarg

Notice that another layer of modules is available now.

You can use shorthand for loading multiple layers of modules:

1
$ module load gnu openmpi orca

Modules with multiple versions available#

RCC offers support for multiple versions of some software, we currently offer many 3.x and 4.x versions of the R package. Running module avail might yield the following:

1
2
3
4
R/3.5.2
R/4.0.0
R/4.1.0 (D)
R/4.2.0

Running module load R will load the default version (currently 4.1.0). But running module load R/4.2.0 will load the environment for that specific version of R.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
$ module load R/4.0.0
$ R --version
R version 4.0.0 (2020-04-24) -- "Arbor Day"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-redhat-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.

$ module load R/4.2.0

The following have been reloaded with a version change:
  1) R/4.0.0 => R/4.2.0

$ R --version
R version 4.2.0 (2022-04-22) -- "Vigorous Calisthenics"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-redhat-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.

Searching for modules#

If you need to search for a specific module, but do not know the hierarchy of modules that it depends on, use the module spider command:

1
2
3
4
5
6
7
8
9
$ module spider ants

------------------------------------------------------------------------------------------------------------------------------------------------------------------
  ants: ants/2.3.5
------------------------------------------------------------------------------------------------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "ants/2.3.5" module is available to load.

      gnu/8.3.1

You can load this module by running the following:

1
$ module load gnu/8.3.1 ants

If the RCC supports multiple versions of the same package, the module spider command will prompt you to specify an exact version to search for:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
$ module spider openmpi

------------------------------------------------------------------------------------------------------------------------------------------------------------------
  openmpi:
------------------------------------------------------------------------------------------------------------------------------------------------------------------
     Versions:
        openmpi/2.1.0
        openmpi/4.1.0
        openmpi/4.1.3

------------------------------------------------------------------------------------------------------------------------------------------------------------------
  For detailed information about a specific "openmpi" package (including how to load the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modules.
  For example:

     $ module spider openmpi/4.1.3
------------------------------------------------------------------------------------------------------------------------------------------------------------------

$ module spider openmpi/4.1.0

------------------------------------------------------------------------------------------------------------------------------------------------------------------
  openmpi: openmpi/4.1.0
------------------------------------------------------------------------------------------------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "openmpi/4.1.0" module is available to load.

      gnu/11.2.1
      gnu/4.8.5
      gnu/8.3.1
      intel/21
      pgi/21.3

    Help:
      "Set paths for GNU OpenMPI compiler & programs."

Unloading modules#

You can unload a module by running module unload MODULENAME; e.g,

1
$ module unload intel

You can unload all modules by running module purge.

Tip

You can also use the shorthand ml command to perform all common functions of the module command:

1
2
3
$ ml              # Equivalent of: module list
$ ml gnu openmpi  # Equivalent of: module load gnu openmpi
$ ml purge        # Equivalent of: module purge

Usage in Slurm submit scripts#

If an executable you need to run on the cluster depends on module(s), you will need to include the module load commands in your submit script before your executable. For example:

1
2
3
4
5
6
7
8
9
#!/bin/bash

#SBATCH -ntasks=8
#SBATCH --job-name='My awesome job'
#SBATCH --time-limit=2-12:00:00
#SBATCH -A backfill2

module load gnu openmpi
srun my_program.sh

Creating custom environment modules#

While RCC staff maintain a robust library of environment modules, you may wish to add custom environment modules for your own jobs. You can create your own custom modules to accomplish this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Create a directory for your custom modules:
$ mkdir ~/modules

# ...create your custom modules here (see details below)...

# Add your module path to the MODULEPATH environment variable:
export MODULEPATH=${MODULEPATH}:${HOME}/modules

# If you want to persist your custom module path after logging out,
add the export MODULEPATH line to the end of your ~/.bashrc file:
echo -e "\n\nexport MODULEPATH=${MODULEPATH}:${HOME}/modules"

A module can be as simple as setting a few environment variables (such as PATH and LD_LIBRARY_PATH) or can be complicated Tcl or Lua scripts. Feel free to refer to the existing system-wide RCC environment modules. These are located at the path: /opt/modulefiles.

Tip

For a simple Lua example, refer to /opt/modulefiles/MPI/gnu/8.3.1/openmpi/4.1.0/orca/5.0.1.lua
For a more involved Lua example, refer to /opt/modulefiles/core/gnu/11.2.1.lua
For a TCL example, refer to /opt/modulefiles/core/autodock