Rescale CLI Tutorial
This section shows different Rescale CLI command functionalities. It shows how to write a job submission script file, submit a job, check job status and shows different ways to download files. The tutorial is based on an LS-Dyna simulation example available from the Rescale resources page.
You can obtain the LS-DYNA simulation file (
neon.refined.rev01.k) used in this example from the example problem.
Input Script and Job Submission
To submit a job that runs on a particular software (LS-DYNA in this case), you need to specify the software in your
#!/bin/bash#RESCALE_NAME="LS-DYNA Neon Example"#RESCALE_ANALYSIS=ls_dyna#RESCALE_CORE_TYPE=Onyx#RESCALE_CORES=2#RESCALE_WALLTIME=1#USE_RESCALE_LICENSEls-dyna -n $RESCALE_CORES_PER_SLOT -i neon.refined.rev01.k -p single
Note: When copying the above script, ensure that there are no spaces following each environment variable. You can find the flags for the input script on the Rescale CLI Commands page. Also, the command on the last line is the same as you would use on the Rescale platform; you can find the command on the Rescale Web UI or using the
list-info command (please refer to the Rescale CLI Commands).
Assuming you have your Rescale API Key environment variable (refer here), run the job using the command:
java -jar /usr/local/bin/rescale.jar submit -i submit.sh
The log will look like this on your terminal:
Running SGE version of Rescale Client App Executing Command. Parsing Input Files No existing files to include No core type specified by user. Using default core type 'Nickel' Found Analysis: ls_dyna Zipping Files Creating temporary encrypted zip at /Users/rescale/lsdyna_job/input.zip Finished writing encrypted file Uploading Files Uploading: /Users/rescale/lsdyna_job/run.sh Uploading run.sh: ##############################| 176B / 176B Uploading: /Users/rescale/lsdyna_job/input.zip Uploading input.zip: ##############################| 9.91MB / 9.91MB Job: Saving Job Job rqRXT: Saved Job rqRXT: Submitting Job rqRXT: --end-to-end flag not set, polling should be done manually.
Job Submission Files
The App will zip up all files in the current working directory into a temporary file called
input.zip. It will then upload both
input.zip to Rescale and include them as input files to the job. It will also include any files listed in the
#RESCALE_EXISTING_FILES line of the input script. For example, if your directory looks like this:
And you submit your job, the App will temporarily create the zip file, as well as parse the
submit.sh script into
run.sh. Your directory will look like this:
input.zip will include the following files -
neon.refined.rev01.k, run.sh, submit.sh. The App will upload the
input.zip file to Rescale along with the
run.sh script. The App will then remove the zip file from the local file system and attempt to save/submit the job. If the job passes validation it will be saved and submitted to Rescale. The App will then start a polling cycle and wait for the status of the job to reach completed. At this point, the App will also print out a <job-id> which can be used for reference.
Checking Job Status
To check the status of the job running, type the following command by replacing
rqRXT with your job ID -
java -jar /usr/local/bin/rescale.jar status -j rqRXT
The terminal will show the status of the message as shown -
java -jar /usr/local/bin/rescale.jar status -j rqRXTAuthenticated as <user>@rescale.comThe status of job rqRXT is Executing
Downloading a File During Job Run
To download a file when the job is running, we use the
download-file CLI command as show below. You can download only one file at a time.
java -jar /usr/local/bin/rescale.jar download-file -j rqRXT -f shared/d3plotjava -jar /usr/local/bin/rescale.jar download-file -j rqRXT -f shared/d3plot
Here, we are downloading the file
d3plot from the run. The command looks for the file in the cluster working directory and hence the file path relative to the working directory must be specified. Here, the file
d3plot is present in the
shared directory. To obtain the relative path of the file, please check the live-tailing window on the Rescale platform as shown.
Downloading Files After Job Has Completed
To download files after a job has completed, we use the
sync CLI command. You can download all the files of the completed job as shown -
java -jar /usr/local/bin/rescale.jar sync -j rqRXT
The terminal will show the following log -
java -jar /usr/local/bin/rescale.jar sync -j rqRXTAuthenticated as firstname.lastname@example.orgSyncing output files
You can use output filtering using
-s flags to download only specific files.
-f flag uses "glob" searching (On client side) while
-s uses "string pattern" searching (On server side). Additionally, you can also use
--exclude to exclude certain file from downloading.
*is a wildcard that will match any string of characters and
? is a wildcard that will match any single character.
Example : The following command downloads all files starting with
d3 while excluding
java -jar /usr/local/bin/rescale.jar sync -j rqRXT -f d3* --exclude d3hsp,d3dump01.0000,d3dump01.0000
For large jobs, the
-s is recommended since it is on the server side.
-s does only string pattern searching and downloads all files with the given string pattern.