Rescale CLI Basics

Basic Submission Syntax

Once the Rescale CLI App is installed, a job can be submitted using the following command:

    rescale-cli submit -p <api-token> -i <input-script>

For users from other platforms (KR, JP, EU), please refer to CLI FAQs for additional flag required while using Rescale CLI.

The <input-script> contains the commands the user may want to run on Rescale. It can be any kind of .sh shell script that can be executed from the command line using the ./ prefix. Regardless of the type of script, the App will try to parse the script to minimize inadvertent issues related to environment setup and un-supported commands. The -p flag is used to read the Rescale API Token you created as shown here.



For using the Rescale CLI commands, the Rescale API key must be used every time. Since the API Key is a long combination of letters and numbers, it will be helpful to create an environment variable and save it in your profile so that you can conveniently use the API variable rather than the actual key every time you use the commands. You can follow these steps for creating the environment variable.

On Linux:

  • On your terminal, type either vi ~/.bashrc OR vi ~/.bash_profile and hit ENTER.

  • This opens up a terminal editor where you can type the following :

      `export RESCALE_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`
    
  • Once you finish typing, hit ESC first and then type :wq to save the changes.

  • You can test the variable by displaying the variable using echo $RESCALE_API_KEY. This will display the Rescale API key on the terminal.

On Windows:

  • On your terminal, enter the following:

      `set RESCALE_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`
    
  • You can double check if the API key has been added to the environment variables by typing

      `echo %RESCALE_API_KEY%`
    

This should display the API key you added.

Let's look at a very simple example.

To run a "Hello World!" job, from a shell terminal (such as in Linux, MacOS, or Cygwin) create a new run directory. Using the text editor of your choice, create a submit.sh input submission script containing:

#!/bin/bash 
#RESCALE_NAME="Hello World" 
#RESCALE_CORES=1 
#RESCALE_ANALYSIS=user_included 
#RESCALE_ANALYSIS_VERSION=0 
#RESCALE_CORE_TYPE=Onyx 
 
echo "Hello World!" > hello_world
sleep 120
 

Note: When copying the above script, ensure that there are no spaces following each environment variable.

Submit the job:

rescale-cli submit -i submit.sh -p <API Key>

If you have added your API Key to the environment variables, you can skip the -p flag in the above command.

This will spin up a new cluster and submit the jobs with the commands from the script. The user can now login to the Rescale Web UI to tail and monitor the job. The job list will show the job as: "Hello World"

Job Submitted

The status page will show the progress of the job and the files can be tailed from within the UI by clicking on the file.

The log will look something like this:

2019-08-26 22:17:55,295 - Authenticated as <user>@rescale.com
2019-08-26 22:17:55,304 - Executing Command.
2019-08-26 22:17:55,308 - Parsing Input Files
2019-08-26 22:17:55,308 - No existing files to include
2019-08-26 22:18:03,112 - Found Analysis: user_included
2019-08-26 22:18:04,274 - No project with the specified name was found: null
2019-08-26 22:18:04,274 - Zipping Files
2019-08-26 22:18:04,276 - Creating temporary encrypted zip at /tmp/rescale-cli-work/input.zip
2019-08-26 22:18:07,054 - Finished writing encrypted file
2019-08-26 22:18:07,055 - Uploading Files
2019-08-26 22:18:07,057 - Uploading: /tmp/rescale-cli-work/run.sh
2019-08-26 22:18:07,058 - Uploading run.sh:
2019-08-26 22:18:07,736 - ##############################| 192B / 192B
2019-08-26 22:18:08,069 - Uploading: /tmp/rescale-cli-work/input.zip
2019-08-26 22:18:08,069 - Uploading input.zip:
2019-08-26 22:18:11,532 - ##############################| 86.39MB / 86.39MB
2019-08-26 22:18:11,818 - Job: Saving Job
2019-08-26 22:18:12,568 - Job kREtT: Saved
2019-08-26 22:18:12,568 - Job kREtT: Submitting
2019-08-26 22:18:13,748 - Job kREtT: --end-to-end flag not set, polling should be done manually.