Application Usage

Psi

Extensive Psi documentation can be found here. Example files can be found on ManeFrame at /grid/software/examples/psi.

Psi4 β5 using GCC-4.9.1

Example Psi4 input file:

#! Optimize FAD B3LYP/aug-cc-pVDZ

molecule fad {
   C -0.114601  1.892903  0.000000
   C  0.114601 -1.892903  0.000000
   O  1.120606  1.459342  0.000000
   O -1.120606 -1.459342  0.000000
   H  1.125662  0.462528  0.000000
   H -1.125662 -0.462528  0.000000
   O -1.120606  1.203465  0.000000
   O  1.120606 -1.203465  0.000000
   H -0.161443  2.982339  0.000000
   H  0.161443 -2.982339  0.000000
}

set basis aug-cc-pVDZ
optimize('b3lyp')
frequencies('b3lyp')

Example submit file:

#!/bin/bash
#SBATCH -J fad                        # Job name
#SBATCH -o fad.log                    # Ouput & error file
#SBATCH -N 1                          # Number of nodes
#SBATCH --mem=18432                   # Available memory on node (MB)
#SBATCH -p parallel                   # Queue (partition)
#SBATCH --exclusive                   # Use entire node exclusively

module purge                          # Unload all modules
module load psi/4.0b5/gcc-4.9.1       # Load Psi module

lfs setstripe -c 8 $SLURM_SUBMIT_DIR  # Set Lustre stripe count
export PSI_SCRATCH=$SLURM_SUBMIT_DIR  # Set Psi scratch directory
export OMP_NUM_THREADS=8              # Set number of OpenMP threads

psi4 fad.inp                          # Run program

netCDF

The netCDF libraries and interfaces and be used to write portable binary files containing well-documented numerical data (either netCDF or HDF files). There are numerous tools availble for writing and accessing data from netCDF and HDF5 files. For large data sets, netCDF and HDF files provide substantial performance and size benefits over text files.

Building small example programs can be done using the following commands.

$ module load netcdf/f_4.4.2 netcdf/c++_4.2.1
$ pgfortran simple_xy_wr.f -I${NETCDF_F_INCLUDE_PATH} -L${NETCDF_F_LIBRARY_PATH} -lnetcdff -o simple_xy_wr
$ pgc++ SimpleXyWr.cpp -I${NETCDF_C_INCLUDE_PATH} -I${NETCDF_CXX_INCLUDE_PATH} -L${NETCDF_CXX_LIBRARY_PATH} -lnetcdf_c++4 -o SimpleXyWr

DMTCP

DMTCP is program that allows for other programs to be arbitrarily check-pointed, meaning that one can restart the program from the checkpoint. Checkpointing is advantageous for two principle reasons. One, jobs that may take longer than the queue allows can be resubmitted repeatedly to the queue and restart from their latest checkpoint. Thus, even calculations that make take a very long time to finish can be run, while still allowing other users consistent and reliable access to cluster resources. Two, if there is there is a failure of some type by the cluster, the job’s progress is preserved.

$ start_coordinator -i 43200
$ dmtcp_launch --with-plugin libdmtcp_infiniband.so --batch-queue --infiniband --ckpt-open-files --modify-env program

The example above allows for all aspects of program to be check-pointed every 12 hours. It is important not to create checkpoints too frequently as this can slow down the calculation’s progress and will generate a significant about of checkpoint data. DMTCP can be used with many single and multi-node program configurations.