Zum Hauptinhalt springen

Cluster Performance Metrics: Generally Available

You can now see how a job uses cluster resources directly on the job status page. Cluster Metrics shows CPU, GPU, memory, storage, and network utilization for any job, with no separate tool or login. Before this, that data only lived on the cluster itself, so checking it meant opening a vendor dashboard or connecting to the machine over SSH.

Release Highlights​

What is it?​

Resource utilization graphs on the job status page, alongside the rest of a job's status:

  • Resource usage where you already look: CPU, GPU (where applicable), memory, storage, and network, on the job status page rather than in a separate tool.
  • Live during the run, and after it: Follow a job's cluster metrics while it runs, then review the same data once it completes.
  • No setup: Collection is automatic. There is nothing to turn on or configure.
  • A record for every job: Metrics are retained, so you can review how a job used the cluster long after it finishes.

Why does it matter?​

Deciding whether a run was sized correctly used to require leaving Rescale. The evidence sat on the cluster, which meant a vendor dashboard or an SSH session, and only while the machine was still up. Once the job finished, the data was gone.

Keeping it on the job page makes two ordinary questions answerable: whether a job that ran slowly was short of a resource, and what hardware the next run should use. Because the metrics are retained, you can also compare a run against an earlier one and catch a performance regression rather than guessing at it.

How is it used?​

Open a job's detail page. Cluster Metrics appears automatically with the rest of the job's status, whether the job is still running or already finished.

Getting Started​

Nothing to configure — collection starts the moment a job runs. See the Cluster Metrics documentation for how to read the graphs.

Known limitations​

  • No data for older jobs: Cluster Metrics only collects data for jobs run after this feature shipped. Jobs that completed before the rollout show no cluster metrics.