Rescale CLI Basics

Basic Submission Syntax

Once the Rescale CLI App is installed, a job can be submitted using the following command:

    rescale-cli submit -p <api-token> -i <input-script>

For users from other platforms (KR, JP, EU), please refer to CLI FAQs for additional flag required while using Rescale CLI.

The <input-script> contains the commands the user may want to run on Rescale. It can be any kind of .sh shell script that can be executed from the command line using the ./ prefix. Regardless of the type of script, the App will try to parse the script to minimize inadvertent issues related to environment setup and un-supported commands. The -p flag is used to read the Rescale API Token you created as shown here.



For using the Rescale CLI commands, the Rescale API key must be used every time. Since the API Key is a long combination of letters and numbers, it will be helpful to create an environment variable and save it in your profile so that you can conveniently use the API variable rather than the actual key every time you use the commands. As an alternative option, a configuration file can be created with your profile to use CLI commands without using the API key everytime. You can follow the steps outlined below to either create an environment variable or create a configuration file with your profile:

On Linux

Setting API Key Environment Variable

  • On your terminal, type either vi ~/.bashrc OR vi ~/.bash_profile and hit ENTER.

  • This opens up a terminal editor where you can type the following :

      export RESCALE_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    
  • Once you finish typing, hit ESC first and then type :wq to save the changes.

  • You can test the variable by displaying the variable using echo $RESCALE_API_KEY. This will display the Rescale API key on the terminal.

Using Profiles in a Configuration File

  • By default, the CLI will look for the profile file in ~/.config/rescale/apiconfig. As a first step, create a rescale directory mkdir ~/.config/rescale. Next, create the configuration file vi ~/.config/rescale/apiconfig and add the following contents:
[default]
apikey = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

The CLI will use the [default] profile automatically. A different named profile can be specified by using the --profile command line argument. It is also possible to use a different config file path by setting the RESCALE_CONFIG_FILE environment variable to the desired path.

If a user is using multiple profiles for different platforms they have to specify the URL to those platforms in the apiconfig as follows:

[default]
apibaseurl = https://platform.rescale.com
apikey = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[eu]
apibaseurl = https://eu.rescale.com
apikey = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

On Windows

Setting API Key Environment Variable

  • On your terminal, enter the following:

      set RESCALE_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    
  • You can double check if the API key has been added to the environment variables by typing

      echo %RESCALE_API_KEY%
    

This should display the API key you added.

Using Profiles in a Configuration File

By default, the CLI will look for the profile file in %USERPROFILE%\.config\rescale\apiconfig. As a first step, navigate to your %USERPROFILE% directory. Create a folder .config and a rescale folder under that. Inside .config\rescale, open notepad and add the following contents:

[default]
apikey = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

While saving the file, type the file name as "apiconfig" (With the double quotes to save it without any file extension) and save as All Files type.

The CLI will use the [default] profile automatically. A different named profile can be specified by using the --profile command line argument. It is also possible to use a different config file path by setting the RESCALE_CONFIG_FILE environment variable to the desired path.

If a user is using multiple profiles for different platforms they have to specify the URL to those platforms in the apiconfig as follows:

[default]
apibaseurl = https://platform.rescale.com
apikey = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[eu]
apibaseurl = https://eu.rescale.com
apikey = XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Let's look at a very simple example.

To run a "Hello World!" job, from a shell terminal (such as in Linux, MacOS, or Cygwin) create a new run directory. Using the text editor of your choice, create a submit.sh input submission script containing:

#!/bin/bash 
#RESCALE_NAME="Hello World" 
#RESCALE_CORES=1 
#RESCALE_ANALYSIS=user_included 
#RESCALE_ANALYSIS_VERSION=0 
#RESCALE_CORE_TYPE=Onyx 
 
echo "Hello World!" > hello_world
sleep 120
 

Note: When copying the above script, ensure that there are no spaces following each environment variable.

Submit the job:

rescale-cli submit -i submit.sh -p <API Key>

If you have added your API Key to the environment variables, you can skip the -p flag in the above command.

This will spin up a new cluster and submit the jobs with the commands from the script. The user can now login to the Rescale Web UI to tail and monitor the job. The job list will show the job as: "Hello World"

Job Submitted

The status page will show the progress of the job and the files can be tailed from within the UI by clicking on the file.

The log will look something like this:

2019-08-26 22:17:55,295 - Authenticated as <user>@rescale.com
2019-08-26 22:17:55,304 - Executing Command.
2019-08-26 22:17:55,308 - Parsing Input Files
2019-08-26 22:17:55,308 - No existing files to include
2019-08-26 22:18:03,112 - Found Analysis: user_included
2019-08-26 22:18:04,274 - No project with the specified name was found: null
2019-08-26 22:18:04,274 - Zipping Files
2019-08-26 22:18:04,276 - Creating temporary encrypted zip at /tmp/rescale-cli-work/input.zip
2019-08-26 22:18:07,054 - Finished writing encrypted file
2019-08-26 22:18:07,055 - Uploading Files
2019-08-26 22:18:07,057 - Uploading: /tmp/rescale-cli-work/run.sh
2019-08-26 22:18:07,058 - Uploading run.sh:
2019-08-26 22:18:07,736 - ##############################| 192B / 192B
2019-08-26 22:18:08,069 - Uploading: /tmp/rescale-cli-work/input.zip
2019-08-26 22:18:08,069 - Uploading input.zip:
2019-08-26 22:18:11,532 - ##############################| 86.39MB / 86.39MB
2019-08-26 22:18:11,818 - Job: Saving Job
2019-08-26 22:18:12,568 - Job kREtT: Saved
2019-08-26 22:18:12,568 - Job kREtT: Submitting
2019-08-26 22:18:13,748 - Job kREtT: --end-to-end flag not set, polling should be done manually.