Rescale CLI Basics
Basic Submission Syntax
With your API token and local copy of the App, a job can be submitted using the following command:
java -jar /usr/local/bin/rescale.jar submit -p <api-token> -i <input-script>
For users from other platforms (KR, JP, EU), please refer to CLI FAQs for additional flag required while using Rescale CLI. The
<input-script> contains the commands the user may want to run on Rescale. It can be any kind of
.sh shell script that can be executed from the command line using the
./ prefix. Regardless of the type of script, the App will try to parse the script to minimize inadvertent issues related to environment setup and un-supported commands. The
-p flag is used to read the Rescale API Token you created as shown here.
Simplifying CLI Commands
You can simplify the rescale commands by creating wrapper scripts. You can also set the RESCALE_API_KEY environment variable (2) or create a profile in the configuration file (3) to omit
-p <api-token> from Rescale CLI commands. These additional steps will help the user in simplifying and using the Rescale CLI commands easily.
Creating a wrapper script
In particular, creating a simple wrapper script will reduce some potential sources of error when submitting a job.
The following is an example contents of a bash wrapper script:
#!/bin/bashjava -jar /usr/local/bin/rescale.jar submit -p <api-token> -E -i $1
<api-token> with the hash string of your API key token. The script also assumes that the Rescale CLI App (
rescale.jar) is located in
/usr/local/bin, so modify accordingly. Because the script will contain your private
<api-token> value, it should be stored in a directory in your
$PATH with limited permissions such as
/usr/local/bin. Also, ensure that the script has the proper permissions so that it can be executed:
chmod +x /usr/local/bin/rescale-submit
Setting the RESCALE_API_KEY enviornment variable
For using the Rescale CLI commands, the Rescale API key must be used every time. Since the API Key is a long combination of letter, it will be helpful to create an environment variable and save it in your profile so that you can conveniently use the API variable rather than the actual key every time you use the commands. You can follow these steps for creating the environment variable -
- On your terminal, type either
vi ~/.bash_profileand hit ENTER.
- This opens up a terminal editor where you can type the following :
bash export RESCALE_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
- Once you finish typing, hit ESC first and then type
:wqto save the changes.
- You can test the variable by displaying the variable using
echo $RESCALE_API_KEY. This will display the Rescale API key on the terminal.
Using profiles in a configuration file
By default, the CLI will look for the profile file in
~/.config/rescale/apiconfig. Create the file (
vi ~/.config/rescale/apiconfig), and add the following contents:
[default]apikey = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
The CLI will use the
[default] profile by default. A different named profile can be specified by using the
--profile command line argument. It is also possible to use a different config file path by setting the
RESCALE_CONFIG_FILE environment variable to the desired path.
Submitting a Test Job - Hello World!
Let's look at a very simple example. Again, this example assumes that you have set up a wrapper script called
rescale-submit shown above.
To run a "Hello World!" job, from a shell terminal (such as in Linux, MacOS, or Cygwin) create a new run directory. Using the text editor of your choice, create a
submit.sh input submission script containing:
#!/bin/bash#RESCALE_NAME="Hello World"#RESCALE_CORES=1#RESCALE_ANALYSIS=user_included#RESCALE_ANALYSIS_VERSION=0#RESCALE_CORE_TYPE=Onyxecho "Hello World!" > hello_worldsleep 120#Create some input files:echo "file 0" > file0mkdir inputsecho "file 1" > inputs/file1echo "file 2" > inputs/file2
Note: When copying the above script, ensure that there are no spaces following each environment variable.
Submit the job:
This will spin up a new cluster and submit the jobs with the commands from the script. The user can now login to the Rescale Web UI to tail and monitor the job. The job list will show the job as: "Hello World"
The status page will show the progress of the job and the files can be tailed from within the UI by clicking on the file:
The log will look something like this:
Running SGE version of Rescale Client App Executing Command. Parsing Input Files No existing files to include Found Analysis: user_included No project with the specified name was found: null Zipping Files Creating temporary encrypted zip at /Users/rescale/helloworld/input.zip Finished writing encrypted file Uploading Files Uploading: /Users/rescale/helloworld/run.sh Uploading run.sh: ##############################| 288B / 288B Uploading: /Users/rescale/helloworld/input. Uploading input.zip: ##############################| 83.08KB / 83.08KB Job: Saving Job Job qMCOdb: Saved Job qMCOdb: Submitting Job qMCOdb: Starting polling cycle Job qMCOdb: Status - Validated Job qMCOdb: Status - Executing Job qMCOdb: Status - Executing Job qMCOdb: Status - Executing Job qMCOdb: Status - Executing Job qMCOdb: Status - Completed Job qMCOdb: Finished... Job qMCOdb: Downloading files to output/ Downloading output/submit.sh Downloading output/run.sh Downloading output/process_output.log Downloading output/inputs/file2 Downloading output/inputs/file1 Downloading output/hello_world Downloading output/file0 . . . Finished downloading files.