Using Dynamic Linking on Darter

Some application requires the use of dynamic linking and shared libraries on the compute nodes. Example of these include software from third-party vendor and popular applications such as Python and OpenFoam. Darter fully supports the use shared libraries and dynamic linking on the compute nodes. All Cray's provided software on Darter were also built with shared libraries support and can be used by dynamically linked application.

Building a Dynamically-linked Application

User can link application dynamically by either setting the environment variable CRAYPE_LINK_TYPE to dynamic or by adding -dynamic flag during linking. The following show some examples on building dynamically linked application and shared libraries.

• Building Hello World program with dynamic linking:

export CRAYPE_LINK_TYPE=dynamic
cc -o hello hello.c

• Same example as above, but using linker flag:

cc -dynamic -o hello hello.c

The same procedures apply to ftn and CC to build Fortran and C++ code with dynamic linking, respectively.

Runtime Considerations

Benefit and Cautionary of Runtime Default

One benefit of dynamically linked application is getting library upgrades without having to recompile. Because libraries for such application are linked during runtime, it is possible that the runtime environment provides newer version of the libraries then the ones used during the building (compiling and linking) of the application. In this case, the application is getting the benefit of automatic library upgrade. On Darter, the runtime environment for dynamically linked application is always the same as the default version shown by the module command for the libraries. For example, since

module avail cray-libsci
---------- /opt/cray/modulefiles ---------
cray-libsci/12.1.3
cray-libsci/12.2.0(default)
cray-libsci/13.0.0

shows ''cray-libsci/12.2.0'' as default, it is also the version loaded during runtime by dynamically linked application by default regardless of the version used during the compiling and linking of such application.

Specifying Runtime Version

Sometime it is desirable to use a specific version of libraries during runtime of a dynamically linked application. This can useful for example during troubleshooting, debugging, and comparing results from different library version. To use a specific version of library during runtime, one must override the default runtime environment by setting ''LD_LIBRARY_PATH'' environment variable. The following example is a batch script that illustrates the steps:

#PBS -l size=32,walltime=1:00:00
#PBS -A UT-ACCOUNT

module swap PrgEnv-cray PrgEnv-gnu
module load fftw/3.3.0.2
module swap cray-libsci cray-libsci/12.1.2
export LD_LIBRARY_PATH=$CRAY_LD_LIBRARY_PATH:$LD_LIBRARY_PATH

aprun -n 32 ./myprogram.exe

In the PBS script above, non-default version of FFTW is loaded, and the default version of cray-libsci is swapped with another version. The module commands set the library path for the specified version for GNU programming environment in the environmental variable $CRAY_LD_LIBRARY_PATH. This is then prepended to the environmental variable LD_LIBRARY_PATH so that the runtime linker can find the correct library paths.

Performance Consideration

The use dynamically linked application does have performance impact. Program startup time may increase due to runtime linking process and symbol lookup in the shared objects, especially with large number of processes. The use of LD_LIBRARY_PATH also introduces another performance impact. There may be some performance penalty in the program runtime as well. Unless it is necessary build an application with dynamic linking, static linking is still recommended for most applications.