PBS Usage

PBS Usage on legion

To submit a script named "pbsjob", enter qsub pbsjob at the command prompt. You can provide directives to PBS on the command line (eg. qsub -q serial pbsjob, or within the script itself (see below for examples).

Note that man pbs and man qsub will provide much more information than is contained here, and also describe many more useful directives which can be issued to PBS.

Queue Structure on legion

There are four queues on legion:

1. serial, which is the default queue. It runs jobs destined for a single processor.
2. exclusive, which is for parallel jobs that require exclusive use of the processors they occupy.
3. parallel, which is for parallel jobs that do not require exclusive processor usage, for example test jobs.
4. omp, which runs jobs on getafix, the Compaq ES-40. This queue is primarily designed for OpenMP jobs.

#PBS -q serial

to run your job in the serial queue. Replace the word "serial" with "exclusive" to run in the exclusive queue, with "parallel" to run in the parallel queue, and with "omp" to run in the omp queue.

Running Parallel Jobs

To successfully run parallel jobs from legion, you need to provide the queue scheduler with the number of nodes that you require.
This is done with the #PBS -l nodes=X directive, where X is the actual number of nodes that you require. If you are using MPI, then you need not provide the "-np" parameter to MPI, as the queuing system does that.

Jobs requiring Extraordinary amounts of Disk Storage

If your job should require more disk than is available on either the /scratch area (1.5 Gb on cnet machines, 4 Gb on bnet machines) or the /bigscratch area (8 Gb), then the anet nodes have >20 Gb available in /scratch. To ensure that your job is run on an anet node, add the PBS directive:

#PBS -l nodes=X:bigdisk (where X is the number of nodes you require [1 for a serial job])