Rescale CLI Advanced

The App uses the SGE input parser by default to parse information from the runtime script to be used as settings on Rescale, and to parse out any unwanted cluster-specific settings in the input submission script (not the wrapper script). For example, the user may have a script named submit.sh that looks like this:

#!/bin/bash 
#RESCALE_NAME="Hello World!" 
module load hello-world
source /home/rescale/environments/hello_world
/usr/lib64/openmpi/bin/mpirun echo "hello-world" > $(hostname)

The parser will try to make sure it parses out the cluster specific line module load... to make sure the script doesn't error on this line. It will then try to remove any commands that change the environment. In this case the lines after source will be parsed out so it doesn't interfere with the Rescale environment. Additionally, the parser will try to remove absolute references to executables/binaries as it is very likely that the executable will not reside in the same location on Rescale as on the local cluster. In this case, the reference to mpirun will be referenced through the PATH CLI directive in the Rescale environment. The resulting script will then be saved to a file named run.sh and will look like this:

#!/bin/bash 
#RESCALE_NAME="Hello World!" 
#module load hello-world 
#source /home/rescale/environments/hello_world 
mpirun echo "hello-world" > $(hostname)

The Rescale analysis command will be set to ./run.sh. This script will be uploaded to Rescale along with the input files.

the App supports the use of Rescale-specific settings directives inside the submit script that allows the user to configure their jobs exactly the way they want it. The currently available variables are listed below. If these variables are not included, the default value will be used.


#RESCALE_CORES=<integer>
{1|2|4|8|16|32|64|128|...}

The number of cores to use. Depending on the core type, the allowed values is a subset of the listed options. The default value for this option is the SGE #$ -pe value.


#RESCALE_CORE_TYPE=<string>
{Marble|Nickel|Onyx|Iron|Gold|Obsidian|...}

The Rescale core type to use. See the Commands section in this document for instructions on how to retrieve a list of valid values for this option.


#RESCALE_NAME=<string>

The name of your job. The default value for this option is the SGE #$ -N value.


#RESCALE_PROJECT_ID=<project name>

Use this when jobs are required to be assigned to a project as set up in the company platform settings.


#RESCALE_ANALYSIS=<string>
{abaqus|adina|aermod|ansys_cfx|ansys_fluent|...|user_included|...}

The analysis used. This should be set to an Analysis "code" value. The Commands section in this document contains instructions on how to retrieve a list of valid values for this option. The default value is any analysis name that can be parsed and is specified in the module or source lines of the script.


#RESCALE_ANALYSIS_VERSION=<string>

The specific version of the selected analysis to use. This is an optional setting. If omitted, the latest version will be used. If specified, this should be set to the "versionCode" value for a specific analysis version. The Commands section in this document contains instructions on how to retrieve a list of valid values for this option.


#RESCALE_EXISTING_FILES=<comma delimited list of strings>

For example: #RESCALE_EXISTING_FILES=<file1-ID>,<file2-ID>,<file3-ID>

This flag will include the listed file IDs if they are already stored on Rescale in this job. Obtaining file IDs is not straight-forward and the user would need access to the API. Please contact Rescale Support (support@rescale.com) for this. If the file IDs listed does not exist, the job will fail on validation. There is no default value for this option.


#USE_RESCALE_LICENSE

Use the Rescale provided licenses for the analysis code. If already existing licenses are to be used, those can be set up using CLI directives, using the execution script.


#RESCALE_ENV_<varname>=<value>

Used to set license server information on the Rescale cluster. Replace with the name of the CLI directive name and with the license server location. For example: #RESCALE_ENV_RLM_LICENSE=8112@license-proxy.rescale.com will create a RLM_LICENSE CLI directive set to 8112@license-proxy.rescale.com on the cluster nodes.


#RESCALE_WALLTIME=<# of hours>

An optional value used to set the maximum number of hours that this job will be allowed to run. Jobs that exceed this number of hours will be terminated. If not specified, the default Max Job Hours value listed in your Rescale user account settings will be used.


#RESCALE_LOW_PRIORITY=<boolean>
{true, false}

An optional value used to run your job in on demand mode. With on demand enabled, jobs may initially enter a queue state until resources become available. Once this on demand job is running, it can be terminated and restarted at most one time. If this happens, jobs may take up to double the runtime. If not specified, the default value is false resulting in Instant mode.

There are some additional advanced settings that can be used to enable SSH access into the master node of the Rescale cluster. Keep in mind that while this can be useful for examining and modifying the results of a job, it negatively impacts the reproducibility of the results should you ever clone and re-run the job in the future.

Currently, there is no way to retrieve the IP address of the master node from the CLI. You will need to use the Job status page on the Web UI to find this information.


#RESCALE_INBOUND_SSH_CIDR=<string>
for example: #RESCALE_INBOUND_SSH_CIDR=50.123.22.112/32

Used to define an IP range that will be allowed inbound SSH access to the master node of the Rescale cluster. If this value is omitted, the default value listed in your Rescale user account settings will be used.


#RESCALE_PUBLIC_KEY=<string>
for example: #RESCALE_PUBLIC_KEY=ssh-rsa AAAA...

Used to set the public key that will be added to the authorized_keys file of the master node on the Rescale cluster. This will allow you to SSH into the master node of the Rescale cluster with the associated private key. If this value is omitted, the default value listed in your Rescale user account settings will be used.

An example script that will both work on SGE and Rescale could look something like this (the module line will be commented out, and the absolute reference to mpirun will be removed):

#!/bin/bash 
#RESCALE_NAME="Converge Sample" 
#RESCALE_CORES=32 
#RESCALE_CORE_TYPE=Nickel 
#RESCALE_ANALYSIS=converge_open_mpi 
module load converge
/usr/bin/mpirun -np 32 converge-2.1.0

An example script that will only work on Rescale could look something like below. The command for the command line can be found in the Analysis Code Table (separate attachment).

#!/bin/bash 
#RESCALE_NAME="Converge Sample" 
#RESCALE_CORES=32 
#RESCALE_CORE_TYPE=Nickel 
#RESCALE_ANALYSIS=converge_open_mpi 
converge-mpi -n all -v 2.1