Compiling code for parallel execution on Kraken must use the proper implementation of MPI, and link against the proper libraries. Many of these complexities can easily be dealt with by using the programming environment, which may be modified using modules.
The programming environment is centered on the compiler. NICS supports C/C++ and Fortran compilers from Portland Group (PGI), INTEL, and GNU. Only one compiler is available at a time to avoid ambiguity. The module command may be used to determine which compiler is being used. Modules which define the programming environment begin with PrgEnv-. To see what programming environments are available on the system you may use the following command:
module avail PrgEnv
In addition, there are modules for the compilers themselves, pgi, intel and gcc. These can be used to change which version of a compiler you are using. If you are changing to a different compiler entirely, you should not need to change these, PrgEnv- should do that automatically.
There are three ways to view release notes for compilers and packages included in the Cray programming environment.
For example, on Kraken you can look for info on pgi version 10.6.0 using the following:
- Using module help pgi/10.6.0 gives information for more recent packages.
- You can find and view the appropriate file in /sw/sources/xt-pe-relnotes
- Release notes are available online at http://docs.cray.com.
Most of the flags for the GNU compilers will not work with PGI. For example, to compile OpenMP code with PGI, you need to use the flag -mp, whereas with GNU you would use -fopenmp. For details on the flags accepted by each compiler, see the man pages for each executable name:
|Fortran||pgf90 or pgf77||gfortran||ifort|
Once the proper module is loaded, the desired compilers are available to be called directly (eg, gfortran test.f). To compile code for the compute nodes, however, it is necessary to use the compiler wrappers provided by Cray. In addition to taking care of cross-compiling, the Cray wrappers look at programming environment and take care of some of the work of linking in libraries, etc.
The following table lists the compiler wrappers available on the system
These wrappers are scripts which gather information from the programming environment, and then call the compilers with the appropriate flags. Any arguments given to the wrapper will be seen by the compiler. For example, cc -g will still create an executable with debugging information.
As you can see using module show PrgEnv-<env> command, some libraries are automatically loaded with the programming environment. Many other libraries are available via modules if you wish to, these may need to be added to the link line with the -l flag. Of course, you can install your own libraries if they are not provided, use the -L dir flag so the compiler can find them.
It should be noted that some libraries link against different library versions depending on the programming environment at the time they are loaded. It is best to change the programming environment first, and then load any additional modules. However, if the programming environment is changed after additional modules are loaded then the module update <modulename> command can be used to reload these modules.
Parallel compiling is common: the make command being, "make -j N" where N > 1. However, the login nodes is not the place for large parallel compilations as it will degrade the performance for all users logged into Kraken. Depending on the circumstances, "make -j 2" would be fine on a login node, assuming the node's load is low. NICS urges users to create an interactive session ("qsub -I") for their large parallel compilation needs.