Compiling code for parallel execution on Kraken can be quite complex. For instance, it must use the proper implementation of MPI, and link against the proper libraries. Many of these complexities can easily be dealt with using the programming environment, which may be modified using modules.
The programming environment is centered on the compiler. NICS supports C/C++ and Fortran compilers from Portland Group (PGI), GNU, and PathScale. Only one compiler is available at a time to avoid ambiguity. The module command may be used to determine which compiler is being used. Modules which define the programming environment (including which compiler is being used) begin with PrgEnv-:
module avail PrgEnv
In addition, there are modules for the compilers themselves, pgi and gcc. These can be used to change which version of a compiler you are using. If you are changing to a different compiler entirely, you should not need to change these, PrgEnv- should do that automatically.
Most of the flags for the GNU compilers will also work for Pathscale, PGI is a little different. For example, to compile OpenMP code with PGI, you need to use the flag -mp, whereas with GNU and Pathscale, you would use -fopenmp. For details on the flags accepted by each compiler, see the man pages for each executable name:
| PGI | GNU | Pathscale | |
|---|---|---|---|
| C | pgcc | gcc | pathcc |
| C++ | pgCC | g++ | pathCC |
| Fortran | pgf90 or pgf77 | gfortran | pathf90 |
Once the proper module is loaded, the desired compilers are available to be called directly (eg, gfortran test.f). To compile code for the compute nodes, however, it is necessary to use the compiler wrappers provided by Cray. In addition to taking care of cross-compiling, the Cray wrappers look at programming environment and take care of some of the work of linking in libraries, etc.
| Compiler Wrappers | |
|---|---|
| C | cc |
| C++ | CC |
| Fortran | ftn |
These wrappers are scripts which gather information from the programming environment, and then call the compilers with the appropriate flags. Any arguments given to the wrapper will be seen by the compiler. For example, cc -g will still create an executable with debugging information.
As you can see using module show PrgEnv-, some libraries are automatically loaded with the programming environment. Many other libraries are available via modules if you wish to, these may need to be added to the link line with the -l flag. Of course, you can install your own libraries if they are not provided on Kraken, use the -L dir flag so the compiler can find them.
It should be noted that some libraries link against different library versions depending on the programming environment at the time they are loaded. It is best to change the programming environment first, and then load any additional modules. The module update (modulename) command can be used to reload all of the modules.
Optimizing
- PGI Compilers & Tools PGI Pemier Support at ORNL – Brent Lebeck
- Cray XT Optimization Basics – Jeff Larkin
