Compute Canada

Fluent

Please note: The FAQ pages at the HPCVL website are continuously being revised. Some pages might pertain to an older configuration of the system. Please let us know if you encounter problems or inaccuracies, and we will correct the entries.

This is a short FAQ on using the computational-fluid dynamics (CFD) code "Fluent" on HPCVL clusters. This software is only licensed for academic researchers who have prior training. The software is only made available to persons who belong to a specific Unix group. See details below.

What is Fluent ?

Fluent is a suite of programs that model systems in computational fluid dynamics (CFD). This includes flows in two- and three-dimensional geometries, and under a variety of conditions: compressible and incompressible; inviscid, laminar and turbulent; Newtonian and non-Newtonian. The analysis can be steady-state or transient. Fluent simulates convective, coupled and radiative heat transfer, it can account for the mixing and reaction of chemical species, and for arbitrary sources of heat, mass, turbulence and momentum. Calculations employ stationary or rotating frames of reference, and a variety of meshes.

Fluent can be used interactively and supplies a graphical user interface. It can also run in batch mode, if the required time for solving a problem is too long for interactive use. The latter situation is the standard if you are using Fluent on HPCVL machines.

The Fluent package consists of several programs:

  • Fluent, the solver;
  • prePDF, a preprocessor for modeling combustion;
  • GAMBIT, a preprocessor for modelling geometries and generating meshes;
  • TGrid, creating volume meshes from boundary meshes;
  • several filters to import meshes from other CAD packages.

Where is Fluent located and how do I access it?

We have two versions of FLuent installed on our systems.

The "legacy" version of Fluent is 6.3. This version is installed to be run on our main production M9000 cluster. Note that this is not a recent versaion of Fluent, but is provided as the last version that is supported on the Solaris/Sparc platform. Note that this version will be de-comissioned in 2014. We recommend to use newer versions if you are just starting off using Fluent. The programs of this Fluent package are located in the directory/opt/fluent/Fluent.Inc/ on sflogin0 and the Solaris nodes.

The most recent version of Fluent is Ansys-14 Fluent. This version is installed to be run under Linux on the "software" (Linux) cluster. It will not work on the Solaris production nodes. The programs of this Fluent package are located in the directory/opt/fluent/ansys-14.0/ on sw0010 and the Linux production nodes.

To use Fluent on any of HPCVL's clusters, you have to be a trained University User of Fluent. It is furthermore required that you read our licensing terms, and sign a statement. We will confirm your statement, and you will then be made a member of a Unix group fluent, which enables you to run the software. Contact us if you are in doubt of whether you qualify to run Fluent on our system.

The Fluent license is "seat limited" and "process limited". At present, there are the following licensing limits on our systems:

  • 25 program runs plus 200 parallel processes on the Solaris platform. This is the "legacy version" 6.3 Fluent. It will be available until June 2014.
  • 25 program runs plus 200 parallel processes on the Linux platform. This is the recent Ansys-14 version of Fluent that is only supported on the Linux platform.

i.e. at most 50 separate sessions can be run simultaneously (serial or parallel). In addition, it is possible to run up to 200 "parallel only" processes in total. One scenario would be 50 users have 5-process parallel jobs running, each with one master (i.e. seat) and 5 slaves (parallel only), thus using up all available Fluent resources.

How do I run Fluent interactively?

The following instructions assume that you are a member of the Unix group fluent. The instructions in this section are only useful if you want to use the graphical user interface of Fluent, for instance to setup a job, or pre- and post-process a production job. If you want to run a production job, please refer to to instructions on how to start a Fluent batch job (see next section).

The Fluent program is started by setting a system variables FLUENT_ARCH and FLUENT_LICENSE_FILE to let the system know about the computing platform employed (lnamd64), the directory where the software resides and the license file used, respectively. It is also useful to include the directory with the Fluent executable in your path. Here is the command sequences to do all this (assuming bash and Ansys/Fluent version 14 on x86/Linux):

export PATH="/opt/fluent/ansys-14.0/v140/fluent/bin:"$PATH
export LM_LICENSE_FILE="/opt/fluent/Fluent.Inc/license/license.dat"
export FLUENT_ARCH="lnamd64"
export FLUENT_LICENSE_FILE="/opt/fluent/Fluent.Inc/license/license.dat"

These settings apply for the 64-bit version of Ansys14 Fluent on Linux, which is the default. 

There is a simple alternative for setting up your environment for the usage of Fluent: you can leave the above commands to usepackage and simply type

use ansys14

on the Linux workup node sw0010. This will automatically issue the above commands. You can also include the use command in your setup file (e.g. .bash_profile) if you are using fluent regularly on the Linux platform.

Then you invoke a graphical user interface by typing

fluent

For this to work you need to be on sw0010 (i.e. the Linux workup node) or another Linux node with interactive access.

The first choice you have to make is if you are solving a two- or a three-dimensional problem, and if you want to do so in single or double precision. You can do so by typing 2d, 3d, 2ddp, or 3ddp, following the fluent command. All commands can be issued in manually or by clicking on the toolbar on the top of the GUI and selecting the appropriate sub-choices. Note that if you want to type a command yourself, and you do not know what your choices are, simply pressing the Enter key will give you a list of applicable commands.

It is of course impossible to even outline how to use Fluent. In many cases, you will want to read in a case file, which has all the required information to describe the system you want to simulate. Such case files have the file extension .cas. Load them by issuing the /file/read-case command or selecting the corresponding menu-commands in the GUI. You can now check and display the grid, specify boundary conditions and material properties, initialize the flow, and perform calculations.

Results are usually saved by the /file/write-case-data command. During an interactive session, it is sometimes a good idea to keep a journal file which records all commands that you have typed in or issued via the GUI. This journal file can later be used as a template for a batch command file. Define the journal file with the /file/start-journal command.

Documentation for Ansys-14 Fluent can be accessed directly from the Fluent GUI by pressing the "Help" button on the upper right (html format), or in pdf format in

/opt/fluent/ansys-14.0/v140/commonfiles/help/en-us/pdf

on the Linux login node sw0010.

Note that the documentation is only accessible if you are signed up as a Fluent user on our system.

How do I set up and run a non-interactive Fluent batch job?

In most cases, you will run Fluent in batch mode. Since you have to have access to Fluent outside of the HPCVL license, most interactive work can be done elsewhere, whereas the computationally intensive runs can be executed on a parallel machine.

For this, you have to set up a batch command file that consists of a sequence of commands that are issue to Fluent. To get an idea how such a batch command file looks, you can produce a journal file during an interactive session, and edit it later to eliminate unnecessary commands. Note that this needs to be done using the command line inside Fluent, not the menu buttons of the GUI. In fact, it is best to generate journal files in sessions that have been started with the -g option, i.e. that do not use the GUI at all.

The "Text User Interface" that has to be used for writing batch files is documented in Chapter 3 of the Fluent 6.3 documentation. In html format this can be found at /opt/fluent/Fluent.Inc/help/html/ug/node48.htm or in pdf format at /opt/fluent/Fluent.Inc/help/pdf/ug/chp3.pdf.  

You also can have a look at a simple example batch file here. The example file (let's call it "example.flin") will read in a case-file from the working directory directory, initialize the flow, and run a single iteration. It then writes out the data on a file fan_1.dat and exits. Note that everycommand has to be included in the batch command file, including the answer "yes" to the question if you really want to exit the program without saving the case file.

Once you have produced a working command file, you can test it by calling

fluent 3d -g -i example.flin

We have assumed you are running a three-dimensional solver in single precision. You will have to alter those entries in different cases. Make sure that the output file for the data (in this case,fan_1.dat) does not exist before you start the job, otherwise the system will query if you want to over-write it and the answer is not in your command file.

Once everything works you could submit this job into the background (using bash) by typing

fluent 3d -g -i example.flin > example.flout 2>&1 &

This would redirect standard output and standard error to example.flout.

Production jobs, especially parallel ones, are submitted on the Sun Fire systems via Grid Engine, which is the subject of the next question.

How to setup/submit a Fluent parallel production job?

To submit a production job on HPCVL clusters, you need to us the load-balancing software Grid Engine. To obtain details, read our gridengine FAQ.

For a Fluent production job, this means that rather than issuing the above batch command directly, you wrap it into a Grid Engine script that looks somewhat like this:

#!/bin/bash
#$ -S /bin/bash
#$ -V
#$ -cwd
#$ -pe shm.pe {number of processors}
#$ -m be
#$ -M {email address for notification}
#$ -o {full name of output file}
#$ -e {full name of error file}
{additional commands, eg for removing files}
. /opt/fluent/ansys-14.0/setup_64bit.sh
fluent {2d/3d/2ddp/3ddp} -t$NSLOTS -g -i {input file}

This script needs to be altered by replacing all the relevant items enclosed in {} by the right values. A Unix text formatet version of the above template can be found here.

The advantage to submit jobs via a load balancing software is that the software will automatically find the resources required and put the job onto a set of processors that have a low load. This will help executing the job faster. Note that the usage of Gridengine for all production jobs on HPCVL clusters is mandatory. Production jobs that are submitted outside of the load balancing software will be terminated by the system administrator.

The Fluent jobs that you will want to run on the HPCVL machines are likely to be quite large. To utilize the parallel structure of our machines, Fluent offers several options to execute the solver in a parallel environment, i.e. on several CPU's simultaneously. The default option for such runs is MPI i.e., it uses the Message Passing Interface for inter-process communication.

To take advantage of the parallel capabilities of Fluent, you have to call the program with additional commandline options that specify the details of your parallel run:

  • -tn where n is the number of processors requested, e.g. if you want to run with 8 processors, you would use the option -t8
  • -g specifies that the GUI should be surpressed. Required for batch jobs.

Parallel jobs should only be run in batch using the Grid Engine. The number of processors specified in our example script appears only once, after

#$ -pe shm.pe

which is where you let the Gridengine know how many processors to allocate to run the program. The internal environment variable $NSLOTS will automatically be set to this value and can then be used in the fluent command line.

It is also necessary to source a setup file called setup_64bit.sh. This will set various environment variables and enable the Fluent program to properly interact with Grid Engine. If you are interested, take a look. The file is readable. 

All processes are allocated within a single node. This is to make communication more efficient and to avoid problems with the control by Gridengine. The effect of this is that, while still using MPI, Fluent employs a so-called shared-memory layer for communication. The disadvantage is that the size of the job is restricted by the number of cores on a node.

Once the script has been adapted (let's call it "fluent.sh"), it can be submitted to the Gridengine by

qsub fluent.sh

from the Solaris login node sflogin0 (which is the GridEngine submit host). Note that the job will appear as a parallel job on the GridEngine's qstat or qmon commands. Note also that submission of a parallel job in this way is only profitable for large systems that use many CPU cycles, since the overhead for assigning processes, preparing nodes, and communication between them is considerable.

There is an easier way to do this: We are supplying a small perl script called that can be called directly, and will ask a few basic questions, such as the name for the job to be submitted and the number of processes to be used in the job. Simply type

AnsysSubmit

and answer the questions. The script expects a Fluent input file with "file extension" .flin to be present and will do everything else automatically. This is meant for simple Fluent job submissions. More complex job submissions are better done manually.

Where can I get further help with Fluent?

Fluent is a complex software package, and requires some practice to be used efficiently. In this FAQ we can not explain it use in any detail.

Online documentation for the programs is available on machines where Fluent is installed. On the HPCVL login node, it can be accessed through a webbrowser by typing:

firefox /opt/fluent/Fluent.Inc/help/index.htm

This should give you an index screen to the html version of the docs. You can also use the pdf documentation in

/opt/fluent/Fluent.Inc/help/pdf/

if you prefer a printable version. To look at specific pdf files, you can use acroread, for instance

acroread /opt/fluent/Fluent.Inc/help/pdf/ug/chp01.pdf

shows Chapter 1 of the User's Guide. Unfortunately, the pdf documentation is somewhat unsystematic and one has to guess from the name of the directories (e.g. "ug" for "User's Guide") what they mean.

The documetnation for Ansys-14 Fluent can be access from inside the program GUI by clicking on the "Help" button on the upper right. This is in html format. The pdf version of the docs can be found in 

/opt/fluent/ansys-14.0/v140/commonfiles/help/en-us/pdf

Fluent documentation is subject to the same license terms as the software itself, i.e. you have to be signed up as a Fluent user in order to access it.

If you are experiencing trouble running a batch command script, check carefully if the sequence of commands is exactly in sync with the program. This might mean typing them in interactively as a test. If you have problems with the GridEngine, read our FAQ on that subject, and maybe consult the manual for that software which is accessible as a PDF file. HPCVL also provide user support in the case of technical problems.