Compiling code for parallel execution on Kraken can be quite complex. For instance, it must use the proper implementation of MPI, and link against the proper libraries. Many of these complexities can easily be dealt with using the programming environment, which may be modified using modules.
The programming environment is centered on the compiler. NICS supports C/C++ and Fortran compilers from Portland Group (PGI), GNU, and PathScale. To avoid ambiguity as to which compiler will be used, only one compiler is available at a time. The module command may be used to determine which compiler is being used:
module list -l 2>&1 | grep PrgEnv
To use a different compiler, use the module command again. The following switches from PGI compilers to Pathscale.
module unload PrgEnv-pgi module load PrgEnv-pathscale
Most of the flags for the GNU compilers will also work for Pathscale, PGI takes some different flags. For example, to compile OpenMP code with PGI, you need to use the flag -mp, whereas with GNU and Pathscale, you would use -fopenmp. For details on the flags accepted by each compiler, see the man pages for each executable name:
| PGI | GNU | Pathscale | |
|---|---|---|---|
| C | pgcc | gcc | pathcc |
| C++ | pgCC | g++ | pathCC |
| Fortran | pgf90 or pgf77 | gfortran | pathf90 |
To see a summary of what a module does, use the module show command. For example, you can see that a PrgEnv-* module changes more than just the compiler. First, it checks to see if any other PrgEnv module is loaded. If there is not, it updates the environment variables that store the programming environment information, and loads several other modules: the compilers (Pathscale requires gcc), a debugger (xt-totalview), MPI library (xt-mpt), math library (xt-libsci), and system modules.
At this point, the desired compilers are available to be called directly (ie, gfortran test.f), however, to take advantage of the programming environment, it is necessary to use the compiler wrappers provided by Cray.
| Compiler Wrappers | |
|---|---|
| C | cc |
| C++ | CC |
| Fortran | ftn |
These wrappers are scripts which gather information from the programming environment, and then call the compilers with the appropriate flags. Any arguments given to the wrapper will be seen by the compiler. For example, cc -g will still create an executable with debugging information.
As you can see using module show, some libraries are automatically loaded with the programming environment. Other libraries are loaded by default. If there are additional libraries that need to be linked in, and they are in the software stack, this can also be done using modules. Libraries not provided on Kraken may be linked in the traditional way, using -L
It should be noted that some libraries link against different library versions depending on the programming environment at the time they are loaded. It is best to change the programming environment first, and then load any additional modules. Do not use the module updatecommand. It currently has a bug, which will cause errors such as the following:
/opt/mpt/3.0.0/xt/GP/bin/mpicxx: No such file or directory
Such problems can be solved by logging out and logging back in. Modules are also used to set up the environment to use programs unrelated to compiling, please see modules.
