How to use the launch script¶

This page describes the eT_launch.py script and its options.

What is the launch script¶

The launch script, eT_launch.py, is a Python script generated during the configure step. After running setup.py, it will be located in your build directory. The behaviour of eT_launch.py does not depend on the location of the script, so you can copy it wherever you want. Note that the testscripts depend on eT_launch.py being in your build directory, so moving it will make the tests fail. The script uses the argparse Python module, making it easy to provide command line arguments.

The script is invoked with the command

path/to/eT_launch.py

and, similarly to setup.py, a list of options can be listed with the --help option.

path/to/eT_launch.py --help

Note

It is not recommended to run your calculations in build. Add eT_launch.py to your path in .bashrc and you can run anywhere.

Below is a description of the default behaviour of eT_launch.py. Almost all of the steps listed can be modified using command line options and/or environment variables.

When running the script, the default behaviour is to create a scratch directory wherever Python feels like. Usually something like /tmp/tmp_kyaaapg.

It will then loop over any files with the .inp extension in the calling directory in an unspecified order. Each input file will then be copied to the scratch directory and eT will be executed as a subprocess.

After eT has finished, the output file, the timing file, and any other relevant output files generated by eT will be copied back to the input directory with a name that depends on the input file. The script will then proceed to the next input file and repeat the process.

Note

After looping over all the input files, the script will delete the scratch directory if all the calculations finished without error messages.

If an error is detected, eT_launch.py will attempt to copy any generated output files to the input directory, exit the loop over input files, and raise an exception without deleting scratch.

For example, if my calling directory contains the files ccsd.inp and cc3.inp before calling eT_launch.py, it should also contain ccsd.out, ccsd.timing.out, cc3.out, and cc3.timing.out afterwards.

All paths given as input or set in environment variables can be given as either absolute or relative to the calling directory.

Warning

If any of the files generated by eT_launch.py already exists, they will be overwritten.

Relevant environment variables¶

This section contains a list of relevant environment variables used by eT_launch.py. All of them can be overridden by command line options.

eT_SCRATCH

Specifies where eT_launch.py should create the scratch directory. For example, if eT_SCRATCH is set to ~/jobs/eT_scratch/, eT_launch.py will create something like ~/jobs/eT_scratch/tmpfel2lrwd. Overridden by --scratch.

eT_SAVE

eT_launch.py can save the files generated by eT required for restart, see the -save, --save-restart-dir option below. If saving is enabled, but no paths given and eT_SAVE is set, eT_launch.py will create a directory based on the name of the output file in the directory specified by eT_SAVE. If eT_SAVE is not set, the directory is created in output directory. Unless otherwise specified, see -of, --output-file, the output directory is the same as the calling directory and the output name is the same as input up to a suffix. For example, if my input file is ccsd.inp, my eT_SAVE is set to ~/store/eT_save, and my launch command is eT_launch.py ccsd.inp -save, then ~/store/eT_save/ccsd_restart_files is created and the restart files are copied there.

AO_BASIS_PATH

Specifies the location where eT will look for basis set files. This variable is rarely required to set explicitly because CMake will set the default to the basis set directory shipped with eT. Overrides the default set by CMake and is again overridden by -basis, --basis-path-dir.

LIBINT_DATA_PATH

When running with libint, libint will look for basis set files here rather than in the AO_BASIS_PATH. The launch script will set these to be the same. This means that if the LIBINT_DATA_PATH is set, it will be overidden by the launch script.

OMP_NUM_THREADS/MKL_NUM_THREADS

Sets the number of OMP threads available for the eT subprocess.

Input options¶

Positional argument¶

input file

The only positional argument of eT_launch.py and can be placed anywhere, except after optional arguments that take a string such as -save, --save-restart-dir, so it is safest to provide this argument first. It accepts any number of strings with paths to eT input files in the order you wish to execute them. The order can be important, especially in restart calculations where you want to restart the second from the first. For example, if I want to run a CCSD and CC3 calculation and restart the CC3 calculation from CCSD, I will use eT_launch.py ccsd.inp cc3.inp. If no input file is explicitly provided, eT_launch.py will look in the calling directory and use all files with the .inp extension (glob(*.inp)) in whatever order python decides. If no files are found, eT_launch.py will stop with an error message.

Note

If an input file is provided as an argument, eT_launch.py will make no references to the calling directory and you can safely call the script from wherever you want.

Optional arguments¶

-h, --help

This is a built in argparse option. Instead of running the script, the list of options and a short description is printed.

-of, --output-file

If you want to name your output files something else than the default, or have them copied to another directory you can use this option. The argument is one or more strings with paths to the eT output file. None is a reserved string causing default behaviour for the corresponding input file.

eT_launch.py input1.inp input2.inp -of None ~/some/dir/output.out

will generate input1.out and input1.timing.out in the input directory and output.out and output.timing.out in ~/some/dir. If ~/some/dir does not exist, it will be created.

If less output files than input files are given, the last output files will get the default names. If more output files are given than input files, the excess is ignored with a warning. Changing the name of the output file with --output-file will change the name of all eT output files.

--omp

Specifies the number of OMP threads set for the eT subprocess. See OMP_NUM_THREADS/MKL_NUM_THREADS above.

--scratch

Specifies which directory to use as scratch. If it does not exist, it will be created.

-ks, --keep-scratch

Tells eT_launch.py to not delete scratch when finished.

Note

On clusters you may not have permission to delete generated scratch directories. Use this option to avoid spam about lacking permissions in your slurm-xxxxx.out file or equivalent.

Default: False

-save, --save-restart-directory

Will make eT_launch.py create a restart file directory based on the output files in the directories provided. If no directory is provided, eT_launch will look for the eT_SAVE environment variable, see eT_SAVE above. If neither a directory or eT_SAVE is set, eT_launch.py will create a directory in the output directory.

None is reserved and will make eT_launch.py use the default save directory while Off will disable saving for the corresponding input file.

Files that can be used to restart eT will be copied to this directory when eT finishes. The files will be copied if eT finishes with an error message, but if you are running on a cluster and the calculation times out, the files will not be saved unless you set --signals or --timeout.

eT_launch.py ccsd.inp cc3.inp -save Off None

Will not save the restart files from the first calculation and will use the default directory based on eT_SAVE or the output file for the second.

Warning

This option is useful on clusters where scratch is deleted automatically. However, the size of the restart files can become very large and you will typically have limited space in your home area. Make sure you have set eT_SAVE to a suitable location or provide one explicitly. Filling up your home quota is an excellent way to get an angry email from your system administrator.

-load, --load-restart-directory

Will make eT_launch.py search for eT restart files in the provided directories and copy them to scratch before starting eT. If there are more input files than load directories, restart files will be copied for the first input files, while the excess is ignored if there are more load directories than input files. None is reserved and disables loading for the corresponding input file.

eT_launch.py ccsd.inp cc3.inp -save ccsd_restart cc3_restart -load None ccsd_restart

Will create the directory ccsd_restart in the output (calling) directory with the restart files after the first calculation and load the files from this directory before starting the second and finally save the restart files in cc3_restart.

-nt, --no-timing

Do not copy the timing file from scratch.

Default: False

-i, --ignore-errors

With this option, eT_launch.py will continue to loop over output files, even if errors are detected from eT or pcm, see -pcm, --pcm-input-file. It will also delete scratch when finished unless --keep-scratch is enabled and exit with nominal status.

Default: False

-t, --timeout

Set a maximum time for eT_launch to run. The argument is a string the parser will parse and look for numbers. The integers will be intepreted as days, hours, minutes and seconds according to the format string. The default format string is %H:%M:%S, hours, minutes, and seconds.

Five minutes before the time runs out, eT_launch.py will terminate the eT subprocess gracefully, copy output files to output directory, and save restart files if requested, see -save, --save-restart-directory.

Note that the parser only cares about numbers in the string and their order.

eT_launch.py -t 10:48:623
eT_launch.py -t "I want to run for 10 hours, 48 minutes, and 623 seconds"
eT_launch.py -t "Lalala10.48 623 trololo"

The above examples will all result in the same time, 10x3600 + 48x60 + 623 - 300 = 39203 seconds.

The time left for copying can be set using -end-t, --end-time, while the time format can be set using -tf, --time-format.

-end-t, --end-time

Sets how long before timeout eT_launch.py should stop eT and start copying and saving files. It uses the same format as -t, --timeout.

Default: 00:05:00

-tf, --time-format

Sets the format used to specify -t, --timeout and -end-t, --end-time. Argument shall be a string containg any of the character combinations %D (days), %H, (hours), %M (minutes) and %S (seconds).

Default: %H:%M:%S

-sig, --signals

Specify signals eT_launch.py should listen for. If the option is enabled, but no signals provided, eT_launch.py will listen for SIGUSR1, provided it is available in the python signals module. See this stackabuse article for a useful discussion of signal handling.

If a signal is detected, eT_launch.py will terminate the eT subprocess, copy any output files, and save any restart files if requested, see -save, --save-restart-directory.

Note

Your workload manager may have the option to send a signal to your job before termination. In Slurm, this can be achieved using the --signal option.

#SBATCH --signal=USR1@60

srun eT_launch.py input.inp -s

In this example, Slurm will send SIGUSR1 to eT_launch.py 60 seconds before the job terminates. Note the use of srun, which is required to get slurm to send a signal to eT_launch.py.

-s, --silent

Silences the prints to stdout from eT_launch.py. Default: False

-bd, --binary-dir

Specifies the directories containing the eT executable for the corresponding input file. Default is set by CMake to the eT build directory. None results in the default executable.

-basis, --basis-path-dir

Specifies where eT should look for basis set files through the LIBINT_DATA_PATH environment variable. Default is set by CMake to the basis set directory shipped with eT.

-pcm, --pcm-input-file

Specifies the PCMSolver input file for the corresponding eT input file and calls PCMSolver in external mode. None disables external PCM for the corresponding input file.