The National Institute for Computational Sciences

Running Jobs

Darter: I have a Python script that uses Python's multiprocessing module for parallelization. How should this be run on Darter?

Python's multiprocessing module is similar to threading, so you should use the following in your Darter batch script to launch the python script on a single node:

module load python
aprun -d 16 python
This will make all 16 cores on the node available to the Python script. Please note: whether or not the cores are fully utilized is up to the programming of the script.

Darter: How can I run my Python script in a batch script?

If the python script is parallelized using MPI (e.g. with mpi4py which is available on Darter), then it should be run just like any other MPI program using the following syntax in your batch script:

module load python
aprun -n numproc python
If there is no MPI in the python script, use the following syntax in your batch script:
module load python
aprun python

General: What are the memory limits on the compute nodes of NICS production resources?

Listed below are the limits on the compute nodes of NICS operated resources. Here are the results from some basic tests that were run on our resources to check the real maximum values for allocatable memory and open files:

System   | MaxMem | MaxOpenFiles
Darter    | 31.1GB | 1015
Nautilus  | 32.1GB | 48
Keeneland | 32.1GB | >2048

Beacon: I was running my program in an interactive job but it didn’t finish. I received the message: qsub: job completed. How do I request more time for interactive jobs?

Request more time for interactive jobs by providing a specific number of hours/minutes/seconds using
qsub –I –l walltime=hh:mm:ss

Note that 24 hours is the maximum that can be requested. If you need an extension, send an email to along with any job ids that need to run for more than 24 hours.

General: Why does my submitted job die with strange shell errors?

The shell initiation line in PBS scripts is not guaranteed to be used to determine the interpreting shell. The default behavior is to use the user's default login shell or the value of the PBS option -S (i.e. #PBS -S /bin/bash or qsub -S /bin/bash). If you are using a shell for a PBS script which is different than your default shell, please use the PBS -S option.


Subscribe to RSS - Running Jobs