US 2024/0403129 A1 · FRAMEWORK FOR SCHEDULING ACCELERATOR-ENABLED WORKLOADS

Abstract

A framework that may be implemented by a workload orchestration platform for scheduling accelerator-enabled workloads on the accelerators in a cluster is provided. In one set of embodiments, the framework enables the platform to schedule accelerator-enabled workloads based on a multitude of user-provided, fine-grained accelerator requirements. In another set of embodiments, the framework enables the platform to automatically recommend an initial set of accelerator resource requirements for an accelerator-enabled workload and automatically right-size such requirements based on telemetry data collected during the workload's runtime.

Claims21

Dependent claims

A method comprising:

receiving, by a computer system acting as a control plane of a workload orchestration platform, one or more accelerator requirements for a workload to be scheduled on an accelerator in a cluster, the one or more accelerator requirements including:
an accelerator type, architecture, or model on which the workload should be scheduled;
an amount of accelerator compute or memory resources that should be allocated to the workload;
and/or
an accelerator service level agreement (SLA) that should be met for the workload;
creating, by the computer system, an accelerator snapshot identifying each accelerator in the cluster and including static and dynamic metadata pertaining to said each accelerator;
determining, by the computer system based on the accelerator snapshot and the one or more accelerator requirements, a target accelerator for the workload;
and
scheduling, by the computer system, the workload on the target accelerator.

6 more elements

A non-transitory computer readable storage medium having stored thereon program code executable by a computer system acting as a control plane of a workload orchestration platform, the program code causing the computer system to execute a method comprising:

receiving one or more accelerator requirements for a workload to be scheduled on an accelerator in a cluster, the one or more accelerator requirements including:
an accelerator type, architecture, or model on which the workload should be scheduled;
an amount of accelerator compute or memory resources that should be allocated to the workload;
and/or
an accelerator service level agreement (SLA) that should be met for the workload;
creating an accelerator snapshot identifying each accelerator in the cluster and including static and dynamic metadata pertaining to said each accelerator;
determining, based on the accelerator snapshot and the one or more accelerator requirements, a target accelerator for the workload;
and
scheduling the workload on the target accelerator.

6 more elements

A computer system acting as a control plane of a workload orchestration platform, the computer system comprising:

a processor;
and
a non-transitory computer readable medium having stored thereon program code that, when executed, causes the processor to:
receive one or more accelerator requirements for a workload to be scheduled on an accelerator in a cluster, the one or more accelerator requirements including:
an accelerator type, architecture, or model on which the workload should be scheduled;
an amount of accelerator compute or memory resources that should be allocated to the workload;
and/or
an accelerator service level agreement (SLA) that should be met for the workload;
create an accelerator snapshot identifying each accelerator in the cluster and including static and dynamic metadata pertaining to said each accelerator;
determine, based on the accelerator snapshot and the one or more accelerator requirements, a target accelerator for the workload;
and
schedule the workload on the target accelerator.

9 more elements

Dependencies are inferred from the claim text.

Framework for scheduling accelerator-enabled workloads

Life of this patent

Framework for scheduling accelerator-enabled workloads

Life of this patent

Abstract

Claims21