Desmond

Simulation software for molecular dynamics

Homepage Version(s): 3.4.0.2

Desmond requires an environment module

In order to use Desmond, you must first load the appropriate environment module:

module load desmond

Desmond is a software developed by D. E. Shaw Research to perform molecular dynamics simulations of biological systems on conventional commodity clusters. Desmond was installed on the FSU HPC cluster as a component of the Schrodinger software suite. The Schrodinger suite also include Maestro, a visualization tool for molecular dynamics. The Schrodinger suite is available on the shared HPC storage system at the following path:

1	`/gpfs/research/software/desmond/schrodinger2013-3`

Using the Schrodinger Suite for the Molecular Dynamics Simulation Process#

The procedure for running a molecular dynamics simulation using Desmond and Maestro can be summarized in the following figure:

In particular, the original structure file imported from the protein database (PDB) needs to be prepared/preprocessed by Maestro to produce the structure file (with force field), which is used as the input of the Desmond simulation. The configuration file contains the simulation parameters such as the global cell, the force field, constraints (if there are any), and the integrator.

Warning

It is not advisable to run GUI software on the HPC login nodes. We suggest you to install Maestro to your personal computer and prepare the structure file on your laptop/desktop, before running the Desmond simulation (the CPU-intensive process) on the HPC cluster.

Command Line Syntax for Desmond#

The syntax for Desmond is as follows:

1	`/gpfs/research/software/desmond/schrodinger2013-3/desmond [Job_Options] [Backend_Options] Backend_Arguments`

The possible [Job_Options] include the following commands:

Syntax	Command Meaning
`-h`	print help message.
`-v`	print version information and exit.
`-WAIT`	don't exit until job completes.
`-p,-NPROC`	number of processors to be used (default is 1).
`-JOBNAME name`	the name of this job.
`-gpu`	run GPU version.
`-jin filename`	files or directories to be transfered to the compute node.
`-jout filename`	files to be copied back to the submit node.
`-dryrun backend_cfg`	generate backend config file only.

The possible [Backend_Options] include the following commands:

Syntax	Command Meaning
`-comm plugin`	use communication plugin (serial or mli)
`-c config_file`	parameter file for simulation.
`-tpp n`	number of threads per processor.
`-dp`	run double precision version (single precision by default).
`-noopt`	do not optimize parameters automatically.
`-overwrite`	overwrite trajectory.
`-profile`	enable backend profiling.

The possible [The Backend_Arguments] include the following commands:

Syntax	Command Meaning
`-in x.cms`	the structure file
`-restore checkpoint`	a check point file for resuming a simulation

(Run the desmond -h command for details.)

As an example, to run the Desmond simulation with input structure file x.cms and configuration file y.cfg on a single machine, you would write out the following input:

1	`$SCHRODINGER/desmond -in x.cms -c y.cfg`

where SCHRODINGER is the environmental variable (path) to the Schrodinger software installation.

Command Line Options for Schrodinger Job Control Facility#

Besides command line options specific to desmond, there are important command line options directly recognized by the Schrodinger Job Control Facility. Here are a few important ones (These options tells the job control facility to run job on a specified host, or submit job to a queue.)

1
2
3

-HOST host
-HOST host:n
-HOST "host_1:n_1 host_2:n_2 ... host_k:n_k"

Here, host is the value of a name entry (not the host entry) in the schrodinger.hosts file (see discussion in the following), or the actual address of a host, and n (n_1, n_2, ... n_k) is the number of cores to the host. When specifying more than one host, use spaces to separate them and enclose them with quotes.

1	`-QARGS queue-args`

This option passes arguments to the queue manager. These arguments are appended to those specified by the qargs settings in the hosts file schrodinger.hosts.

1	`-TMPDIR directory`

This option specifies the scratch directory for the job. The job directory is created as a subdirectory of the scratch directory. We suggest you to use $HOME/scratch or $HOME/_tmp.

There are some options to see the configured runtime for Desmond. For example,

-ENTRY

This option shows the section of the schrodinger.hosts file that will be used for this job provided the -Host host option points to a section of the hosts file.

Remark: Command-line options always take precedence over the corresponding environment variable.

Running Desmond Simulation on the HPC Cluster#

Molecular dynamics simulations are CPU-intensive. A Desmond simulation can run on the HPC cluster in two ways:

Via the Slurm job submit script (recommended)
Through the Schrodinger's Job Control Facility

Submitting Desmond Jobs Using a Slurm Script#

Below is an example Slurm submit script:

#!/bin/bash
  #SBATCH -J desmond_mpi
  #SBATCH --mail-type=ALL
  #SBATCH -N 2
  #SBATCH --ntasks-per-node=8
  #SBATCH -t 24:00:00
  #SBATCH  --mem-per-cpu=2000 # 2GB of memory per CPU * 8 = 16GB total
  #SBATCH -p genacc_q
  # the desmond module defines the environmental variables needed 
  # for running desmond       
  module load desmond
  #$SCHRODINGER=/gpfs/research/software/desmond/schrodinger2013-3 
  mpirun $SCHRODINGER/desmond -in  x.cms  -c y.cfg -comm mpi

where x.cms and y.cfg are respectively the structure file and simulation parameter file.

Submitting jobs Using the Schrodinger Job Control Facility#

The Job Control Facility obtains information about the hosts on which it will run jobs (and other information needed by the queue) from the hosts file, schrodinger.hosts, and will submit/monitor the job for you. The default hosts file is located in:

1	`$SCHRODINGER/schrodinger.hosts`

An example entry of the schrodinger.hosts is as follows:

name:   genacc_q
host:    hpc-login
queue:  slurm
qargs:  -p genacc_q  -N 2 --ntasks-per-node=8  --mem-per-cpu=2000 -t 14:00:00:00
processors: 500
tmpdir: $HOME/_tmp # You will need to create this directory before running the job

In this example:

genacc_q is the name of the entry (each entry is one job submission scenario)
submit is the name of the host (the login node of the HPC cluster)
slurm is the queuing software
qargs contains the command line options of qsub (or sbatch for Slurm). In the genacc_q queue, the wall clock limit is 14 days
tmpdir is the scratch space for Desmond application (you can use for example $HOME/scratch for your own schrodinger.hosts file).

To submit a parallel job using the above configuration, input the following:

1 2	`$ module load desmond $SCHRODINGER/desmond -in x.cms -c x.cfg -HOST genacc_q`

The -HOST genacc_q flag above will tell the Schrodinger Job Monitoring Facility to use the entry with name genacc_q in the schordinger.hosts file to create the job submit script. Consequently, the simulation will be submitted to the queue genacc_q asking for 2 nodes and 8 cores each).

To use your own hosts file, create a directory .schrodinger under your home directory, and copy the file below to it and edit it:

1 2	`$ mkdir $HOME/.schrodinger $ cp $SCHRODINGER/schrodinger.hosts $HOME/.schrodinger/schrodinger.hosts`

You also need define the environmental variable SCHRODINGER_HOSTS to point to this file:

1	`export SCHRODINGER_HOSTS=$HOME/.schrodinger/schrodinger.hosts`

Monitoring Jobs Using the Schrodinger Job Control Facility#

Besides the job control commands provided by the job scheduler such as squeue, sbatch, etc., Desmond simulations can be monitored by the Schrodinger Job Control Facility. The Job Control facility provides tools for monitoring and controlling the jobs that it runs. Information about each job is kept in the user's job database. This database is kept in the directory $HOME/.schrodinger/.jobdb.

A Desmond job can be manipulated and monitored by the Schrodinger utility jobcontrol as follows:

1	`$SCHRODINGER/jobcontrol [command or query]`

In the above, command is the command for the action you want to perform, and query defines the scope of the action performed by the command.

The possible command options include:

Syntax	Command Meaning
`-cancel`	Cancels a job that has been launched but not started
`-kill`	terminates a job immediately
`-list`	lists the JobID, job name and status
`-resume`	continues running a paused job
`-dump`	shows the complete job record

The possible query options include:

Syntax	Command Meaning
`all`	all jobs in your job database
`active`	all active jobs (not finished)
`finished`	all jobs finished, or a JobID. The JobID is a unique identifier consisting of the name of the submission host, a sequence number, and a hexadecimal timestamp, e.g., submit-0-a1b2c3d4.

For example, to list all the jobs in your job database that finished successfully, enter:

1	`$SCHRODINGER/jobcontrol -list finished`

To list just the job whose JobId is submit-0-a1b2c3d4, enter:

1	`$SCHRODINGER/jobcontrol -list submit-0-a1b2c3d4`

To list the complete database record for a job, enter the command:

1	`$SCHRODINGER/jobcontrol -dump jobid`