Welcome to HPCVL -- Install FAQ

Install FAQ

This is a short introduction about "How To Setup" your account on the HPCVL SunFire systems. When first logging in, you will be presented with a default setup that enables the use of basic system commands, compilers, a parallel runtime environment, and the Grid Engine scheduling software. This help file is meant to give you a basic idea about how to modify that default.

Questions:

What's with those shells?
How do I set up my shell?
How do I use "use"?
How to do this manually?
How do I get started with program development?
How do I run pre-installed software?
How do I run parallel code?
How to get my terminal to work with this system?
How do I use a "secure shell"
Still doesn't work! Where to get help?

Answers:

What's with those shells?

When you apply for an account you are presented with the choice between different login shells:

bash ("Bourne again" shell).
csh (C shell)
ksh (Korn shell)

They differ in their "feel" and how they handle system commands. ksh and bash largely understand each other's command syntax. csh is somewhat different.

bash is a "GNU" shell that uses features from the older shells csh and ksh. It is quite popular because it offers many convenient features, such as command and file-name completion by pressing the "Tab"-key. Its scripting syntax is almost identical to the one of ksh, which enables you to use ksh setup files.

Some people find csh best because it is simple and has a syntax that is similar to the C programming language (thus the name). A lot of script files for Unix are written in csh, and it is a good idea to maintain a good setup file for csh, even if csh is not your login shell. This will make your life easier when you are trying to install something or to run shell scripts from third parties.

ksh offers some extended features compared with csh. As an example, you can recall commands that you typed previously. With ksh, this is usually done by pressing the "escape" key, and scrolling through previous commands with arrow keys. However, its syntax is rather different, so that csh commands can usually not be issued to ksh.

Choosing a login shell is largely a matter of taste. However, some programs require a specific shell to run. You can always call a shell by typing "/usr/bin/name_of_shell", where name_of_shell might be csh, tcsh, ksh, bash or sh (the latter is another standard shell that is often used for scripts). Make sure you maintain setup files for the shells you use, even if they are not your login shell.

In scripts, shells are usually called with a "shebang" sequence, that looks like:
#!/bin/csh
in the first line. In such cases, it can happen that environment variables, such as PATH get reset to some default. This is why it is a good idea to start csh scripts with
#!/bin/csh -f. The -f option avoids sourcing in the .cshrc file, leading to consistent behaviour of the script for different users (with different .cshrc files).

All shells are documented in the man pages: type "man name_of_shell" to get more information than you'd care to know.

How do I setup my shell?

To a large degree, your shell is set up using a tool called usepackage. This allows you to replace the setting of several environment variables, and the execution of setup scripts by a simple command of the form use package, where package stands for a pre-defined application or feature that you want to include in your setup. We discuss the use of use in the next section.

On a lower level, there are several setup files in your home directory. Depending on the shell you use, these files differ in their structure and command syntax. If you are a new user of our systems, the system will copy a set of default files into your home directory. If you need setup files for a shell other than your login shell, or you are not a new user, but want to set yourself up from scratch, the default files can be found in /etc/skel. Check out the README file in that directory.

For csh there are two basic setup files: .cshrc and .login. The former gets "sourced" (i.e. applied to the shell) every time you call a shell, while the latter only applies to login shells. Due to the leading "." these files are not visible if you type "ls"; you will have to type "ls -la" to see them. Editing this file lets you change the basic behavior of your shell. If you don't have any setup files in your account, our system will apply a simple default setup that lets you access system functions, compile code, run parallel programs, and use our scheduling software. Note that there may be other setup files. csh sources the .login file when you log in, and the .logout file when you log out. All these files reside in your home directory.

For ksh the main setup file is called .profile. It is also "invisible". If you don't have it in your directory, your shell will still be set up "reasonably" by our system. You can also obtain a default .profile from /etc/skel.

Since bash uses a similar scripting syntax as ksh, it can execute .profile on login as well. Other setup files for bash are .bashrc and .bash_profile which have a similar function for bash as .cshrc and .login do for csh. For details about which files are sourced in which order, consult the man pages: man bash or equivalent. As for the other shells, bash setup files may be obtained from /etc/skel.

The setup files are mainly used to set environment variables such as PATH, and to execute a few commands when the shell is started. Note that all shells first execute a default setup script, and therefore have default values for the most common variables. You can check the current values of all environment variables by typing "env".

How do I use "use"?

Most setup consists of the setting of environment variables. These are variables that are used by your shell to determine its desired behavior. Other setup operations might include the execution of scripts. To simplify this task, our system supplies the use command. The manual setting of environment variables (such as PATH, for instance) is replaced by typing:
use package_name
where package_name stands for an application or functionality that you want to include in your setup. A list of possible package_names can be obtained by typing:
use -l

Here is an example:
In order for you to be able to use the programs in the Blastwave distribution, you would need to add the /opt/csw/bin directory to your PATH variable, the /opt/csw/share/man directory to your MANPATH variable, and the /opt/csw/share/info directory to your INFOPATH variable. Instead of doing this, you can also just type
useblastwave. Note that Blastwave is included in the default user setup, and is used here only as an example.

The use command will also work inside of your login setup file, e.g. .login for a csh. This makes it possible to apply the setting permanently, and you won't have to retype the use command every time you log in (which would defeat the purpose). By default, users are supplied with a reasonable setup that corresponds to
use standard-user-settings, and includes access to most system functions, Cluster Tools, Grid Engine, Sun Studio compilers, and a few "extras".

However, if you invoke a shell from the command prompt or by executing a script, the new shell will not know about the use command. You either have to setup that shell "manually", or source a file to teach the shell about use. This can be done with a single line:
source /opt/usepackage/share/usepackage/use.csh (for csh, tcsh, etc.)
. /opt/usepackage/share/usepackage/use.ksh (for sh, ksh, etc.)
. /opt/usepackage/share/usepackage/use.bsh (for bash)
Note that it is rarely necessary to make invoked shells aware of use, as they inherit the setup of the invoking (login) shell.

How do I do this manually?

If you don't want to use use, you can of course apply the settings directly. This means essentially setting a few basic environment variables, the most important of which is PATH, which tells the system where to look for the commands that you issue or for the files that are used while executing something within a shell. You want to make your shell aware of some directories with system commands and shell commands in them.

Another environment variable that is often useful is MANPATH. This is for the Unix manual pages, and tells the system where to look for online-documentation.

Yet another one is LD_LIBRARY_PATH, which Is sometimes used by applications to find dynamic runtime libraries. The variable tells the runtime environment where to look for these, and your program might not run if LD_LIBRARY_PATH is not set correctly. If you experience problems with missing runtime libraries, try playing with LD_LIBRARY_PATH, otherwise it's best left unset.

For a csh, the the command to set environment variables is setenv:
setenv VARIABLE VALUE
where VARIABLE is the name of the environment variable, and VALUE is the value it is set to. This might be a number, or a string, in which case it should be enclosed by double quotes.

For ksh or bash the command is just the binary operator '=':
VARIABLE=VALUE,
followed by the "export" command, which makes the variable part of the environment:
export VARIABLE
Note that for bash it is possible to place "export" in front of the variable assignment instead of issuing two separate commands:
export VARIABLE=VALUE

To access the value of a environment variable, you have to type a "$" in front of it. For example you want to see which value your variable PATH has, you type:
echo $PATH
where echo is just a standard Unix command, and "$PATH" returns the value of PATH. The following command will append something to a previously defined variable (in a csh):
setenv PATH=$PATH":/yet/another/directory"
Here, "PATH" denotes the variable and "$PATH" denotes its present value.

Sometimes a variable needs to be reset for a specific application. It is then best to write a shell script that sets the variables and starts the application, rather than setting the variables globally in your startup files.

You can consult the configuration files of usepackage to find out which setting you need to apply to run a specific software or access certain features. The configuration is in
/opt/usepackage/etc/usepackage.conf
The syntax in that file is not hard to read, for instance the entry

>> blastwave : "Blastwave Solaris Packages" <<
blastwave * SunOS 5.* : 
PATH += /opt/csw/bin,
MANPATH += /opt/csw/share/man,
INFOPATH += /opt/csw/share/info ;

tells us what directories to prepend to the environment variables PATH, MANPATH, and INFOPATH, respectively if we want to use "Blastwave".

How do I get started with program development?

The proper configuration for usage of the Sun Studio compiler suite, as well as Sun Cluster Tools (which enables parallel programming with MPI), are already included in your default setting. If you want to use a different set of compilers, for example revert to a previous version, you can use the use command. For instance, if the default setting is studio12, you can change to the previous version 10, by typing "use studio10". For different packages that you might want to use, run use -l.

How do I run pre-installed software?

The SunFire grid has a lot of software installed on it. Some of this software requires specific license agreements, other programs are freely accessible. With the use command, most of them can be set up with a single line such as "use fluent" in your shell's startup file. If the software you want to run is not included in our usepackage list, please contact us, and we can include it. If you are using very specific software that is not accessed by other users, you might have to do the setup manually.

Here is a few steps to follow in that case.

a) Check out the documentation for the specific program, including users' manuals and home pages.

b) Inform yourself about licensing. Some software requires each individual user to hold a license, some is covered by a collective license agreement, some does not require a license at all. For example, the computational-fluid-dynamics code "Fluent" requires all users to be covered by their own license, whereas the license agreement for the electronic-structure code "Gaussian" is site-bound, and covers all users for a specific machine. Finally, code such as "Gamess" (another quantum-chemistry program) are free to use by all users, although the distributor encourages registration.

c) Set the proper environment variables. This can usually be done in your shell setup files, since you'll be running the same code on most occasions you log on. These variables might include the PATH, but also variables specific for the program in question. Which ones to set you will be able to find out in most cases from the program documentation. Remember that this is only necessary if no entry exists in the usepackage

configuration file, whcih can be checked by running "use -l".

How do I run parallel code?

That depends on how the code is "parallelized". If it was "multi-threaded" by the compiler (automatic or via compiler directives), it is usually enough to set the environment variable PARALLEL or OMP_NUM_THREADS to the number of threads that should be used.

If it is MPI code, a special parallel runtime environment has to be used. The command there is mprun, which has command-line options that let you tell how many and which processors to use. This command is part of the CRE (Cluster Runtime Environment). CRE involves a good deal of commands that let you submit, cancel and check jobs that use MPI. The settings for this are included in our default setup.

How do I get my terminal to work with this system?

In many cases the machine used to connect to our grid is not a SUN, so an environment variable has to be set to tell our system what kind of terminal you are using. This variable is TERM. Often the system sets TERM to a "reasonable" value automatically, sometimes this has to be done by hand.

For instance, if you log in from a "telnet" window under Windows, your terminal type is usually a vt100 (or vt220, with minor differences between them). In this case you will want to type "setenv TERM vt100" for a csh or "export TERM=vt100" for a ksh or bash. If you are always logging in from the same terminal, you might as well put these commands into your setup file (.chsrc or .profile or .bashrc). If you are lucky, you don't have to issue them at all because the SUN figures it out by itself.

A common type of terminal used in an X-window desktop environment is "xterm". If you are logging in from an IBM work station running AIX, you should use "xterm" as well, since no one outside of IBM knows what an "aixterm" is. This is a case where the automatic detection won't help you.

If you want to run graphics applications that create windows on your desktop you will have to set another environment variable called "DISPLAY". That is done like:
"setenv DISPLAY=ip_address:0.0" for a csh and "DISPLAY=ip_address:disp_num; export DISPLAY" for the others. ip_address stands for the IP address of your terminal or desktop, and disp_num is it's "display number" (which is quite often 0). If you don't know it, you can usually find it by typing "who am i" on sfnode0. Setting this environment variable will then display any newly called interfaces on your terminal. Or will it? ... Not quite. First you have to issue a command "xhost" on your local machine (usually a desktop). Typing "xhost sfnode0.hpcvl.queensu.ca" on your terminal enables you to receive displays from remote machines. (Note that sometimes you have to search a little for "xhost" on you own machine; on Sun's it's under /usr/openwin/bin). Now you're ready.

Note that the above procedure of setting the DISPLAY environment variable can most often be avoided by configuring your connection software (see next section about ssh clients) to enable "X11 forwarding".

How do I use a "secure shell"?

A secure shell (ssh) is a connection protocol that allows logging into a remote machine with an encrypted data stream. This makes it virtually impossible for third parties to "eavesdrop" and gain access to information such as passwords or transfered data. The command associated with a secure shell is ssh.

The use of secure software for interactive login and data transfer is HPCVL user policy, and therefore mandatory. Unencrypted protocols such as telnet or ftp are disabled, i.e. we do not supply a server for them.

Most Unix system include ssh and it's file-transfer equivalent sftp. To log into our system using ssh, simply type
ssh -X username@sfnode0.hpcvl.queensu.ca
where username is your login ID. The -X has the effect of enabling the forwarding of graphics, thus making the explicit setting of a DISPLAY environment variable unnecessary, and encrypting the data stream associated with the graphics as well. This "X11 forwarding" can usually be enabled in the configuration of the whole ssh client.

There is a variety of terminal programs that offer ssh for Windows PC's. One of them is called PuTTY and is available for free. It is a simple program that works on all Windows system reliably. Note that if you use a Windows machine to log in you will also need a XWin emulator program such as Xwin32 to display graphics on your desktop.

Still doesn't work. Where can I get help?

A lot of SUN specific documentation can be found at http://docs.sun.com. If you have questions that you can't resolve by checking documentation, you can call or send email to one of our user support people. We will respond as quickly as possible. If you don't get a reply to your email within a day or two, call on the phone. If we can't help you immediately, we will contact SUN Microsystems for further assistance.