Welcome to HPCVL -- NWChem FAQ

NWChem FAQ

This is a short "Frequently Asked Questions" file on using the parallel electronic-structure code "NWChem" on the HPCVL SunFire cluster. This software uses MPI as a message-passing system and is (in principle) able to run on an arbitrary number of processors. Its ability to perform a very broad spectrum of molecular-structure calculations, ranging from CI to ab-initio molecular dynamics, makes it an interesting alternative to the standard electronic structure code Gaussian.

Questions:

1.  What is "NWChem"?
2.  Where is the program located and how do I access it?
3.  How do I use it?
4.  Can I run the program in parallel?
5.  How do I set up and submit "NWChem batch jobs on the GridEngine?
6. Where can I get further help?
7.  How is NWChem licensed?

Answers:

1. What is "NWChem"?

NWChem is an electronic-structure code that is suitable to perform complex calculations on molecular structure. It was specifically designed to perform well on high-performance parallel computers. The installation on the SunFire cluster of HPCVL employs the MPI message passing package for parallel execution.

NWChem allows, among others, the following calculations to be performed:

Hartree-Fock (e.g. RHF, UHF, ROHF etc.)
DFT including spin-orbit DFT, with many exchange and correlation functionals.
Complete Active Space SCF (CAS-SCF)
Coupled-Cluster (CCSD, CCSD+T, etc.)
limited CI (eg, CISD) with perturbative corrections
MP2 (2nd-order Mollar-Plesset Perturbation Theory)
In general: single-point calculations, geometry optimizations, vibrational analysis.
Static one-electron properties, densities, electrostatic potentials.
ONIOM model for multi-level calculations on larger systems
Relativistic corrections (Douglas-Kroll, Dyall-Dirac, spin-orbit)
Ab-initio molecular dynamics (Carr-Parinello)
Extended (solid-state) systems DFT
Classical force-fields (Molecular Mechanics: AMBER, CHARMM, etc.)

For a more complete list, see the official NWChem homepage and click on "capabilities".

2. Where is "NWChem " located and how do I access it?

The NWChem program is located in the directory /opt/nwchem/bin. To access it, you have to either include this directory in your path:

export PATH='/opt/nwchem/bin:'$PATH (for ksh)
setenv PATH '/opt/nwchem/bin:'$PATH (for csh)

or (preferably) use the usepackage command

use nwchem

which will set you up automatically. You also need to have a NWChem setup file in your home directory. The best way to do this is to create another symbolic link

ln -s /opt/nwchem/data/default.nwchemrc $HOME/.nwchemrc

Note that the .nwchemrc file is "invisible" and may be checked out with the 'ls -la' command. A final thing that has to be done before you can run NWChem is to make sure the environment variable LD_LIBRARY_PATH includes the 64-bit high-performance libraries. You can check the variable by typing

echo $LD_LIBRARY_PATH

and if the directory /opt/SUNWhpc/lib/sparcv9 is not in there, include it by typing

export LD_LIBRARY_PATH='/opt/SUNWhpc/lib/sparcv9':$LD_LIBRARY_PATH (ksh)
setenv LD_LIBRARY_PATH '/opt/SUNWhpc/lib/sparcv9':$LD_LIBRARY_PATH (csh)

The above setting will be applied automatically when you type use nwchem. Of course, these commands may be included in your setup files, so you don't have to type them each time you run NWChem. It might also be necessary to reset the LD_LIBRARY_PATH variable after using NWChem, particularly if you want to run MPI applications that use 32-bit addressing.

3. How do I run "NWChem"?

Before you can access NWChem, you need to read our license agreement and sign a statement that you have done so and will abide by it. See the last section for details.

Like other electronic-structure programs, NWChem is run by supplying an input file that defines the system on which to perform a calculation (usually a molecule, or a group of molecules), and the method to use (i.e., the level of calculation, such as "Hartree-Fock", the basis set, and other details of the computation).

The variety of possible calculations is great, and so is the complexity of systems. It is impossible for us here to explain the format that a NWChem input file needs to have. This is explained in the User's Manual which is available for download in pdf and postscript format.

We provide a simple sample input file which should be given the file extension .nw. This extension may then be omitted when calling the program. NWChem creates typically a whole array of output files that are documented in the User's Manual. A general log is displayed on the console, and may be saved in a file by redirecting the standard output:

nwchem sample > sample.log

Where we assume that your input file is called sample.nw and you want to save the log to a file sample.log.

Note that this is just the basic syntax of the program call. In practise you will use a parallel environment to execute the program (see next section). In fact, attempts to execute NWChem by just typing the above line will fail.

4. Can I run NWChem in parallel?

Yes, in fact you have to. NWChem is inherently parallelized and designed to scale well on a multi-processor machine or a cluster. The underlying parallel system is MPI (Message Passing Interface) which is a commonly available standard that runs on many platforms. Consult our Parallel Programing FAQ and follow some of the links in there if you want to have more information about MPI. Even if you want to use only one processor for your NWChem run (which sometimes is the best solution, particularly for smaller computations), you have to submit the program to a parallel environment. In the case of our SunFire cluster, this environment is called CRE (Cluster Runtime Environmment), and the relevant command is mprun:

mprun -np 8 nwchem sample > sample.log

This will run your sample.nw input file on eight processors. If you want to run the program in serial, you have to use the '-np 1' option in the above command line. Note that you are only allowed to run NWChem this way for small test systems! For any production jobs, you have to submit the task to the GridEngine queuing software (see next section).

5. How do I set up and run a NWChem batch job?

Production jobs are submitted on the SunFire systems via the GridEngine, which is a load-balancing software. To obtain details, read our GridEngine FAQ . For an NWChem batch job, this means that rather than issuing the command in the previous section directly, you wrap it into a GridEngine batch script.

For an example for such a batch script please click here. This script needs to be altered by explicitly replacing the entries enclosed in {}'s. We suggest you use it as a template for all your NWChem runs. For details, consult our GridEngine FAQ . Also note that there is no need in this script to redirect the standard output via the > operator. Instead, you define where the output goes to the GridEngine via the "#$ -o" command. The script is submitted to the GridEngine by typing

qsub batch_file_name

The advantage to submit jobs via a load balancing software is that the software will automatically find the resources required and put the job onto a set of processors that have a low load. This will help executing the job faster. Production jobs on our cluster must be submitted using GridEngine from the normal login node (sfnode0), and executed under GE's control on the other (production) nodes hpcvl0-hpcvl9, without any need for you to log in.

6. Where can I get further help?

NWChem is a very complex software package, and requires practice to be used efficiently. In this FAQ we can not explain it use in any detail. Complete documentation for the program is available in the form of the User's Manual , which is an absolute must-have if you want to use this program. It is available at the official NWChem website . This website is also your number-one address if you have questions about the program or encounter problems. They feature a very useful FAQ and even a tutorial.

There is also an email list server for NWChem users, that serves as a user support forum. To subscribe to the user's list, send an email message with the line

subscribe nwchem-users@emsl.pnl.gov your@email.address

to majordomo@emsl.pnl.gov.

If you are experiencing trouble running a batch command script or with the GridEngine, read our FAQ on that subject, and maybe consult the manual for that software which is accessible as a PDF file. HPCVL also provide user support in the case of technical problems. Contact us, we might be able to help, or pass you on to someone who can.

7. How is NWChem licensed?

NWChem is obtainable free of charge from the Pacific Northwest National Laboratory. To obtain your own copy, go here and click on the "Download" tab.

The distributors of NWChem require their users to sign a User's Agreement, and HPCVL has done so. Like with other software, HPCVL requires users who want to use NWChem, to read this agreement , and sign a statement that they have done so and will abide by the terms of the license. You can fax the signed statement to us. You will then be included in a unix user group that has access to the NWChem executables.