Compute Canada

How do I write and submit batch jobs?

Please note: The FAQ pages at the HPCVL website are continuously being revised. Some pages might pertain to an older configuration of the system. Please let us know if you encounter problems or inaccuracies, and we will correct the entries.

To run a job with grid engine you have to submit it from the command line or the GUI. But first, you have to write a batch script file that contains all the commands and environment requests that you want for this job. If, for example, test.sh is the name of the script file, then use the command qsub to submit the job:

qsub test.sh 

And, if the submission of the job is successful, you will see this message:

your job 1 (``test.sh'') has been submitted.

After that, you can monitor the status of your job with the command qstat or the GUI qmon.

When the job is finished you will have two output files called "test.sh.o1" and "test.sh.e1".

Now, let's take a look at the structure of a Grid Engine batch job script. We first recall that a batch job is a UNIX shell script consisting of a sequence of UNIX command-line instructions (or interpreted scripts like perl,...) assembled in a file.

And in Grid Engine, it is a batch script that contains additionally to normal UNIX command special comments lines defined by the leading prefix ``#$''.

The first line of the batch file starts with

#! /bin/bash

which is default shell interpreter for Grid Engine. But you can force Grid Engine to use your preferred shell interpreter (bash for example) by adding this line at your script file

#$ -S /bin/bash

to tell GE to run the job from the current working directory add this script line

#$ -cwd 

if you want to pass some environment variable VAR (or a list of variables separated by commas) use the -voption like this

#$ -v VAR or #$ -V 

The former sets a specific variable, while the latter passes all variables listed in env.

Insert the full path name of the files to which you want to redirect the standard output/error respectively (the full pathname is actually not necessary if the #$ -cwd option was used).

#$ -o file for standard output #$ -e file for standard error

The prefix #$ has many options and is used the same way you use qsub, so check qsub man pages to take a look at those options.

Here is a serial sample script that has to be modified to fit your case. All entries enclosed in {} must be replaced.

Insert an email-address after #$ -M for notification. It is best to use Username@localhost with Username replaced by your actual user name and place file named .forward that contains your real email address into your home directory. This way, your email address remains private and invisible to other users.

Note that that qsub usually expects shell scripts, not executable files. To submit the job you simply type

qsub serial.sh

Note that from the command line you can issue options and type, for instance

qsub -cwd -v VAR=value -o /home/tmp -e /home/tmp serial.sh