# US 2021/0173719 A1 · Dynamic autoscaler for cloud platform

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0173719 A1
- Document ID: US20210173719A1
- Status: Published application
- Kind: A1
- Application number: 16/569,603
- Filed: 2019-09-12
- Published: 2021-06-10
- Earliest claimed: 2019-09-12
- Assignees: Pivotal Software, Inc.
- Applicants: Pivotal Software, Inc.
- Inventors: Jacques Chester; John Feminella
- Web page: https://patentgrep.com/patent/US20210173719A1

## CPC

- G06F 9/5083: {Techniques for rebalancing the load in a distributed system}
- G06F 9/455: Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F 9/45533: {Hypervisors; Virtual machine monitors}
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 9/48: Program initiating; Program switching, e.g. by interrupt
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/50: Allocation of resources, e.g. of the central processing unit [CPU]
- G06F 9/5005: {to service a request}
- G06F 9/5011: {the resources being hardware resources other than CPUs, Servers and Terminals}
- G06F 9/5016: {the resource being the memory}
- G06F 9/5022: {Mechanisms to release resources}
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 9/505: {considering the load}
- G06F 9/5061: {Partitioning or combining of resources}
- G06F 9/5072: {Grid computing}
- G06F 11/30: Monitoring
- G06F 2009/45583: {Memory management, e.g. access or allocation}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing autoscaling of computing resources in a distributed computing system. One of the methods includes receiving, by an autoscaler of a cloud computing platform, a plurality of capacity parameters for a workload that is to be executed in one or more instances of the cloud computing platform or to consume some amount of computing resources. An autoscaling process is repeatedly performed that computes a capacity cost for a workload based on a current utilization measure for multiple workloads of the cloud computing platform. A target capacity level is computed that quantifies the amount of computing resources that should be allocated to the workload to achieve the target service level for the computed demand forecast, resource allocation is automatically adjusted for the workload according to the target capacity level.

## Claims

### Claim 1

A method performed by a distributed computing system comprising a plurality of computers, the method comprising:
receiving, by an autoscaler of a cloud computing platform, a plurality of capacity parameters for a workload that is to be executed in one or more instances of the cloud computing platform or to consume an amount of computing resources; and
repeatedly performing an autoscaling process comprising:
computing a capacity cost for a workload based on a current utilization measure for multiple workloads of the cloud computing platform;
computing a delay cost for the workload and a capacity lead time for the workload, wherein the capacity lead time represents an expected amount of time to allocate additional resources to the workload;
computing a target service level for the workload based on the delay cost and the capacity cost;
computing a demand forecast for the workload;
computing a target capacity level that quantifies the amount of computing resources that should be allocated to the workload to achieve the target service level for the computed demand forecast; and
automatically adjusting resource allocation for the workload according to the target capacity level.

### Claim 2 (depends on 1)

The method of claim 1, wherein the target service level represents a ratio of the delay cost to a sum of the capacity cost and the delay cost.

### Claim 3 (depends on 1)

The method of claim 1, wherein the delay cost represents a cost of the delay resulting from demand not being met with currently available capacity.

### Claim 4 (depends on 1)

The method of claim 1, wherein the capacity lead time is based on a forecast or user-specified workload lead time that quantifies the time required for the created resources to become operational or to become usable by the workload.

### Claim 5 (depends on 1)

The method of claim 1, wherein the capacity lead time represents an expected amount of time to launch another instance of the workload.

### Claim 6 (depends on 1)

The method of claim 1, wherein the capacity lead time represents a length of time required for the cloud computing platform allocate to the workload resources which are already partly or wholly provisioned without having been allocated to the workload.

### Claim 7 (depends on 6)

The method of claim 6, wherein the capacity lead time represents a length of time required for the cloud computing platform to launch an instance of the workload on a computing resource where the software is already provisioned on disk.

### Claim 8 (depends on 6)

The method of claim 6, wherein the capacity lead time represents a length of time required for the cloud computing platform to launch an instance of the workload on a computing resource where the software is already provisioned in memory.

### Claim 9 (depends on 8)

The method of claim 8, wherein the capacity lead time represents a length of time required for the cloud computing platform to launch an instance of the workload on a computing resource where the software is already provisioned in memory as a process that is not currently schedulable but that is otherwise known to and managed by a kernel of the computing resource.

### Claim 10 (depends on 1)

The method of claim 1, wherein computing the target capacity level comprises:
computing a demand forecast for the capacity lead time; and
computing a target capacity level to achieve the target service level according to the demand forecast for the capacity lead time.

### Claim 11 (depends on 1)

The method of claim 1, further comprising performing the autoscaling process for multiple workloads, with each workload having a different respective delay cost, capacity forecast, capacity lead time, demand forecast, or observed demand.

### Claim 12 (depends on 1)

The method of claim 1, wherein performing the autoscaling process comprises automatically scaling down resources for the computing workload.

### Claim 13 (depends on 12)

The method of claim 12, wherein automatically scaling down resources for the computing workload comprises automatically scaling down the workload when the workload has a lower target service level or a lower delay cost than other workloads on the cloud computing platform.

### Claim 14 (depends on 1)

The method of claim 1, wherein performing the autoscaling process comprises automatically scaling down resources for the computing workload when the total demand for computing resources across all workloads increases.

### Claim 15 (depends on 1)

The method of claim 1, wherein performing the autoscaling process comprises automatically scaling up resources for the computing workload.

### Claim 16 (depends on 15)

The method of claim 15, wherein automatically scaling up resources for the computing workload comprises automatically scaling up the workload when the workload has a higher target service level or a higher delay cost than other workloads on the cloud computing platform.

### Claim 17 (depends on 1)

The method of claim 1, wherein performing the autoscaling process comprises automatically scaling up resources for the computing workload when the total demand for computing resources across all workloads decreases.

### Claim 18

A distributed computing system comprising a plurality of computers and one or more storage devices storing instructions that are operable, when executed by the plurality of computers, to cause the plurality of computers perform operations comprising:
receiving, by an autoscaler of a cloud computing platform, a plurality of capacity parameters for a workload that is to be executed in one or more instances of the cloud computing platform or to consume some amount of computing resources; and
repeatedly performing an autoscaling process comprising:
computing a capacity cost for a workload based on a current utilization measure for multiple workloads of the cloud computing platform;
computing a delay cost for the workload and a capacity lead time for the workload, wherein the capacity lead time represents an expected amount of time to allocate additional resources to the workload;
computing a target service level for the workload based on the delay cost and the capacity cost;
computing a demand forecast for the workload;
computing a target capacity level that quantifies the amount of computing resources that should be allocated to the workload to achieve the target service level for the computed demand forecast; and
automatically adjusting resource allocation for the workload according to the target capacity level.

### Claim 19 (depends on 18)

The system of claim 18, wherein the capacity lead time represents a length of time required for the cloud computing platform allocate to the workload resources which are already partly or wholly provisioned without having been allocated to the workload.

### Claim 20

One or more non-transitory computer storage media encoded with computer program instructions that when executed by a distributed system comprising a plurality of computers cause the plurality of computers to perform operations comprising:
receiving, by an autoscaler of a cloud computing platform, a plurality of capacity parameters for a workload that is to be executed in one or more instances of the cloud computing platform or to consume some amount of computing resources; and
repeatedly performing an autoscaling process comprising:
computing a capacity cost for a workload based on a current utilization measure for multiple workloads of the cloud computing platform;
computing a delay cost for the workload and a capacity lead time for the workload, wherein the capacity lead time represents an expected amount of time to allocate additional resources to the workload;
computing a target service level for the workload based on the delay cost and the capacity cost;
computing a demand forecast for the workload;
computing a target capacity level that quantifies the amount of computing resources that should be allocated to the workload to achieve the target service level for the computed demand forecast; and
automatically adjusting resource allocation for the workload according to the target capacity level.
