Skip to content

Submitting GPU jobs

The HPC supports GPU jobs. The HPC includes an increasing number of compute nodes equipped with GPU hardware. Currently, we have NVIDIA GeForce GTX 1080 Ti GPU cards, but we are working on adding more to our cluster.

GPU nodes are available for general use on the backfill2 partition. In addition, if your department, lab, or group has purchased GPU resources, they will be available on your owner-based Slurm account.

Submitting GPU Jobs#

If you wish to submit a job to node(s) that have GPUs, simply add the following line to your submit script:

1
#SBATCH --gpus=[1-4] # <-- Choose a value between 1 and 4 cards

Nodes contain two to four GPU cards. Specify the number of GPU cards per node you wish to use after the --gpus directive. For example, if your job requires four GPU cards, simply specify 4:

1
2
#SBATCH --gpus=4          # This job will reserve four GPU cards in a single node.
#SBATCH --gpus-per-node=4 #

Full Example Submit Script#

The following HPC job will run on a GPU node and simply print information about the available GPU cards:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
#!/bin/bash

#SBATCH --job-name="gpu_test"
#SBATCH -n 1
#SBATCH --mail-type="ALL"
#SBATCH -t 1:00

# Here is the magic line to ensure we're running on a node with GPUs
#SBATCH --gpus=1

# If your owner-based partition has access to GPU nodes, you can use that. 
# For general access users, GPU jobs will run only on backfill.
#SBATCH -p backfill2

# Not strictly necessary for this example, but most
# folks will want to load the CUDA module for GPU jobs
module load cuda

# Print out GPU information
/usr/bin/nvidia-smi -L

Your job output should look something like this:

1
2
3
4
GPU 0: GeForce GTX 1080 Ti (UUID: GPU-dc0def06-a6a8-e652-a626-967ca59ea0cd)
GPU 1: GeForce GTX 1080 Ti (UUID: GPU-cdb555e0-dce2-52c6-1029-361375ed79ce)
GPU 2: GeForce GTX 1080 Ti (UUID: GPU-a8606fee-ac2b-4763-fb0d-2f5c9f75244a)
GPU 3: GeForce GTX 1080 Ti (UUID: GPU-433b5155-9a11-055a-5970-770545ae6264)

For more information and examples, refer to our CUDA software documentation

Last update: October 25, 2022