"module" is special software that enables you to set up quickly software for use. The main advantage is that several version of the same package can cohabit without interfering with each other that way.
module is available on all 4 possible systems at BlueFern:
- power7 AIX (kerr)
- power7 Linux (beatrice)
- BlueGene/P (foster)
- Visualisation cluster (viz0-viz5)
module works by changing environment variables that influences where to find software, where libraries are located and where to find documentation. It may also define additonal environment variables to help you perform compilation or use the package.
First module needs "bash" to work properly. This is the default when you log on the BlueFern system, while it is possible to use other shells we do not currently support it. module can be used to load various piece of software.
- we need to know what is available. The command is avail
The listing tells us what modules are available and where they are located. Note that the last group comes under the heading "TESTING" and is probably not ready for general use. You are welcome to try and report any problems you have with them but we are not responsible for any damages to your work resulting from their use.
module names have two parts. The first part is the software (or suite of software) name and the second is a version number usually followed by an information extension. Common extension are as follow:
- xl: compiled with IBM XL compiler (xlc, xlC or xlf*)
- xlf: compiled with IBM xlf (or xlf90) fortran compiler
- xlf_ext: compiled with IBM xlf and the option -qextname. This is useful for software calling fortran functions from C/C++ code and expecting the GNU naming convention for these functions (R does this for example).
- gnu43: compiled with GNU 4.3.4 (4.3.2 on BG/P) compilers
- gnu48 compiled with GNU 4.8.x compilers
- poe: uses IBM poe for MPI
- openmpi: uses openmpi for MPI
- serial: suitable for serial use (no MPI)
- We can list the currently loaded modules
- To load modules
Using "add" is an alternative to "load". Note that in the list of available modules some have "(default)" next to them in that case only specifying the first part of the module name will load this particular version
You can load several modules at once:
To remove a module:
Using "rm" is an alternative to "unload". Note that you don't need to give a version number in that case. It is usually impossible to load several versions of the same module.
To remove all loaded modules:
Most modules will define or update the following variables to enable access to the software or help access to libraries:
- PATH: executables can now be called directly and mask default versions if there are any
- MANPATH: man page for the package will be made accessible if there are any
- INFOPATH/INFODIR info pages will be accessible if there any
- PKG_CONFIG_PATH if you need to compile software against a package providing .pc pkgconfig will be able to find them
- CPATH when using GNU C/C++ include files for the software will be found without needing extra "-I" statements. gfortran may not find .f03 headers. IBM XL compiler do not recognize this variable at the present stage
- LIBRARY_PATH when using GNU compiler libraries for the the software will be found without needing extra "-L" statements. IBM XL compiler do not recognize this variable at the present stage
- LD_RUN_PATH when compiling simple program with GNU or xlc/xlC compilers (but not xlf), runpaths will be automatically set so that LD_LIBRARY_PATH is not needed to run the program
Other variables created for software including libraries. If you load pkgname/some_version the following variables will be created:
- PKGNAMEINC path to include files
- PKGNAMELIB path to libraries
For example on loading any hdf5 module the variables HDF5INC and HDF5LIB are created. A simple use would be:
Most modules don't define the variable LD_LIBRARY_PATH . Instead libraries and executable have been compiled with run paths pointing to the right libraries. This means the executables and libraries know where to find their dependent libraries. For example if we look at mdrun_mpi from gromacs 4.6.3 compiled with openmpi and located in /usr/local/pkg/gromacs/4.6.3_openmpi_gnu48/bin:
We can see that without loading any modules mdrun_mpi has no undefined libraries and from readelf we can see all the paths that are searched to find libraries..
Defining LD_RUN_PATH and using it during compilation helped achieve this result and it will work in simple compilation.
When is setting LD_RUN_PATH not helpful: when the building system for your software sets run paths itself. This happens frequently in three cases:
- compiling with xlf, xlf90, xlff95 xlf2003 and sometimes xlC. The xl fortran compiler sets run paths of its own to find its run time, this over ride LD_RUN_PATH.
the building system uses libtool (usually used in the configure/make make install routine). libtool set its own run paths as well. When configuring add the following
at the end of your configuration line.
the build system uses cmake. cmake does weird stuff to run paths. When configuring with cmake (or ccmake) call it the following way
Some modules depends on other modules. On beatrice the loading of dependencies is automatic for all modules. The only issue is in case of conflicts with already loaded modules. If you start a completely new task and you need a new set of module you may find it easier to start by purging ("module purge")your current set of modules before loading the one you want.
Using modules with loadleveler
As mentioned earlier we only support using module with bash. So your loadleveler script needs to use bash
Loading modules modifies and creates environment varaibles. When using poe or openmpi to run jobs on more than one node your script is executed on a master node and then a launcher will start more instances on the other nodes (remote nodes). By default the launcher just log in the remote nodes and tries to start the extra instances of you MPI program. The environment variables are not, by default, forwarded to the remote nodes. Most modules using openmpi will set an alias for mpirun that will handle the variables correctly. If you still encounter a problem using "mpirun -x PATH" instead of just plain "mpirun" may help.
If you are using IBM poe try adding the following line to your loadleveler script:
In most case starting the application with its full path will work: