Skip to content

Python on the HPC

A very powerful, easy to use and object-oriented scripting language.

Homepage Version(s): 3.9.25 or any (see below)

Python is a general-purpose programming language widely used for scientific computing, data analysis, machine learning, automation, and web development.

The Python ecosystem includes packages for numerical computing, visualization, machine learning, bioinformatics, workflow automation, and many other research tasks.

Using Python on the HPC#

Currently, Python version 3 is available on the HPC:

1
2
$ python --version
Python 3.9.25

Using newer versions of Python on the HPC#

The RCC includes the most recent Python version from the default vendor-provided package repository (3.9.25 at the time of writing).

If you need a newer version of Python, we recommend using uv to install it. Below is a brief example of installing the latest version (3.14 at the time of writing). 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Load the uv module
module load python-uv

# Deactivate any existing Conda environments (only for conda users)
conda deactivate

# Create a new virtual environment with the latest Python version
uv venv ~/myenv --python 3.13

# Activate the new virtual environment
source ~/myenv/bin/activate

# Check Python version
python --version

If you need a specific version, you can specify it when creating the virtual environment with the --python MAJOR.MINOR flag. For example, to create a virtual environment with Python 3.8: 

1
[USER@h22-login-26 ~]$ uv venv ~/myenv --python 3.8

Python environments#

Our recommended way to use Python on the HPC is to create a custom environment using either uv, virtualenv, or Conda & Anaconda. All three tools are available and pre-installed on the HPC.

If your script uses only the Python standard library and does not require third-party packages, you can use the default system Python.

uv#

To use uv, refer to our dedicated uv page.

Virtualenv#

The simplest way to create an isolated Python environment is to use Python's built-in venv module. Each environment contains its own Python executable and package directory and is isolated from the system-wide Python installation.

The following example shows how to create a Python virtual environment named myapp in the default version of Python:

Creating a Virtualenv#

1
2
3
4
5
6
7
8
# Create a Python virtualenv named 'myapp'
$ python -m venv ~/myapp

# Use pip3 to install packages (e.g., numpy)
~/myapp/bin/pip3 install numpy

# Run Python in virtual environment
~/myapp/bin/python3

Activating a Virtualenv#

As you can see from the above, you need to specify the full path each time you execute a Python-related command in your new Virtualenv. If you want to make your Virtualenv Python environment the default, you can run source ~/myapp/bin/activate:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# Make myapp Python commands the default
source ~/myapp/bin/activate

# You will know that the virtualenv is activated, because it will prepend the prompt 
(myapp) [~]$  

# Now, you can be sure that any commands reference the 'myapp' runtime
$ which python
~/myapp/bin/python

$ which pip
~/myapp/bin/pip

To deactivate the environment, type deactivate at the prompt:

1
2
3
(myapp) $ deactivate
$ which python
/usr/bin/python

Tip

If you know that you will be using your custom Python virtualenv exclusively, or most of the time, you can autoload it upon login by appending the activate script to your .bashrc file: 

1
echo -e "\n\n# Activate myapp virtual environment\nsource ~/myapp/bin/activate" >> ~/.bashrc

Conda#

To use Conda, refer to our dedicated Conda page.

Jupyter Notebooks#

To use Jupyter Notebooks, our recommended development environment for Python collaboration, refer to our dedicated Jupyter Notebooks page

Python HPC Jobs#

You submit Python jobs to the HPC the same way you would with other jobs. Be sure to include the activation commands for any virtual environments or Conda environments.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
#!/bin/bash

#SBATCH -n 1
#SBATCH -A genacc_q
#SBATCH -t 01:00:00
#SBATCH --mail-type=ALL

# Here, we load any environment modules we need (recommended)
module load python-uv

# Here, we activate our virtual environment (substitute the appropriate uv or conda command) 
source ~/myapp/bin/activate

# Here we perform the execution of our script
python my_python_script.py

Using Python v2 on the HPC#

The Python community sunsetted support for Python 2 in 2015. We recommend that you upgrade your libraries to Python 3. However, if you need to run Python 2, you can use PyEnv:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
# 1. Download and install pyenv (this installs to ~/.pyenv)
curl -fsSL https://pyenv.run | bash

# 2. Add the following to your ~/.bashrc file (use the command below or open ~/.bashrc in your preferred text editor)
cat >> ~/.bashrc <<'EOF'

# Load PyEnv
export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init - bash)"

EOF

# 3. Reload the shell (or logout and login again)
source ~/.bashrc

# 4. Test that it's working
which python
~/.pyenv/shims/python # <-- Expected output

# 5. Install Python 2 (takes a few minutes)
pyenv install 2
Downloading Python-2.7.18.tar.xz...
-> https://www.python.org/ftp/python/2.7.18/Python-2.7.18.tar.xz
Installing Python-2.7.18...
patching file configure
patching file configure.ac
patching file setup.py
patching file Mac/Tools/pythonw.c
patching file setup.py
patching file Doc/library/ctypes.rst
patching file Lib/test/test_str.py
patching file Lib/test/test_unicode.py
patching file Modules/_ctypes/_ctypes.c
patching file Modules/_ctypes/callproc.c
patching file Modules/_ctypes/ctypes.h
patching file Modules/_ctypes/callproc.c
patching file setup.py
patching file Mac/Modules/qt/setup.py
patching file setup.py
Installed Python-2.7.18 to /gpfs/home/cam02h/.pyenv/versions/2.7.18

# 6. Activate the Python 2 environment
pyenv shell 2.7.18

# 7. Verify that you are using Python v2
python --version
Python 2.7.18

You can specify a specific version of Python if you need to run a version older than 2.7.18:

1
2
3
4
5
# Install
pyenv install 2.6.11

# Activate
pyenv shell 2.6.11

Until you activate a specific Python version using pyenv shell, pyenv shims the system-wide default Python installation (/usr/bin/python; v3.9). So, it is safe to keep in your ~/.bashrc.

If you need to load Python2 for a Slurm job, you can add pyenv shell [version] in your submit script before the executable.

1
2
3
4
5
6
7
#!/bin/bash
#SBATCH directives...

# Use Pyenv to activate 
pyenv shell 2 

python my_python_script.py