# US 11,307,895 B2 · Auto-scaling cloud-based memory-intensive applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,307,895 B2
- Document ID: US11307895B2
- Status: Granted
- Kind: B2
- Application number: 16/737,099
- Filed: 2020-01-08
- Published: 2020-07-09
- Granted: 2022-04-19
- Earliest claimed: 2019-01-09
- Assignees: University of Utah Research Foundation
- Applicants: University of Utah Research Foundation
- Inventors: Joe H. Novak; Sneha K. Kasera; Ryan Stutsman
- Web page: https://patentgrep.com/patent/US11307895B2

## CPC

- G06F 9/5016: {the resource being the memory}
- G06F 9/4418: {Suspend and resume; Hibernate and awake}
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 9/505: {considering the load}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 9/5088: {involving task migration}
- G06F 9/54: Interprogram communication
- G06F 2009/45562: {Creating, deleting, cloning virtual machine instances}
- G06F 2009/45583: {Memory management, e.g. access or allocation}
- G06F 2209/5011: Pool
- G06F 2209/5019: Workload prediction
- G06F 2209/5022: Workload threshold
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Improved techniques for dynamically responding to a fluctuating workload. Resources are reactively scaled for memory-intensive applications and automatically adapted to in response to workload changes without requiring pre-specified thresholds. A miss ratio curve (MRC) is generated for an application based on application runtime statistics. This MRC is then modeled as a hyperbola. An area on the hyperbola is identified as satisfying a flatten threshold. A resource allocation threshold is then established based on the identified area. This resource allocation threshold indicates how many resources are to be provisioned for the application. The resources are scaled using a resource scaling policy that is based on the resource allocation threshold.

## Claims

### Claim 1

A computer system that reactively scales resources for memory-intensive applications and that automatically adapts to workload changes having no pre-specified thresholds, said computer system comprising:
one or more processors; and
one or more computer-readable hardware storage devices having stored thereon computer-executable instructions that are executable by the one or more processors to cause the computer system to at least:
generate a miss ratio curve (MRC) of an application based on application runtime statistics that are extracted from the application;
model the application's MRC as a hyperbola;
identify an area on the hyperbola where the MRC is determined to satisfy a flatten threshold;
establish a resource allocation threshold based on the identified area on the hyperbola, the resource allocation threshold indicating how many resources are to be provisioned for the application; and
scale the resources that are to be provisioned for the application using a resource scaling policy that is based on the resource allocation threshold, wherein using the resource scaling policy includes (i) launching a newly provisioned virtual machine (VM) to satisfy a memory requirement of the application and (ii) while the VM is being launched, using a cloud function (CF) as an interim resource to satisfy the memory requirement of the application until such time as the VM becomes ready,
wherein the CF is a lightweight computing element available in a cloud platform, and
wherein the CF is short lived and refrains from maintaining state when decommissioned.

### Claim 2 (depends on 1)

The computer system of claim 1, wherein generating the MRC of the application is performed during runtime of the application and is performed by capturing memory references from a memory slice of the application while the application executes.

### Claim 3 (depends on 2)

The computer system of claim 2, wherein capturing the memory references from the memory slice is performed using a virtual machine (VM).

### Claim 4 (depends on 1)

The computer system of claim 1, wherein scaling how much memory is allocated to the application is performed by refraining from requiring prior knowledge of an expected workload of the application.

### Claim 5 (depends on 1)

The computer system of claim 1, wherein the application runtime statistics are extracted using a hypervisor, which collects memory traces.

### Claim 6 (depends on 1)

The computer system of claim 1, wherein the memory allocation threshold determines a memory size that avoids thrashing and storage-bound performance.

### Claim 7 (depends on 1)

The computer system of claim 1, wherein the resources include a virtual machine (VM).

### Claim 8 (depends on 1)

The computer system of claim 1, wherein scaling the resources that are to be provisioned for the application includes temporarily using a cloud function (CF) to temporarily absorb an increase in a workload of the application during an intervening time period in which a virtual machine (VM), which has been tasked with providing additional resources to the application, is coming online.

### Claim 9

A method for reactively scaling resources for memory-intensive applications and for automatically adapting to workload changes with no pre-specified thresholds, said method comprising:
generating a miss ratio curve (MRC) of an application based on application runtime statistics that are extracted from the application;
modeling the application's MRC as a hyperbola;
identifying an area on the hyperbola where the MRC is determined to satisfy a flatten threshold;
establishing a resource allocation threshold based on the identified area on the hyperbola, the resource allocation threshold indicating how many resources are to be provisioned for the application; and
scaling the resources that are to be provisioned for the application using a resource scaling policy that is based on the resource allocation threshold, wherein scaling the resources that are to be provisioned for the application includes:
determining a new virtual machine (VM) is to be launched to increase total system memory in response to an increased workload of the application;
initiating launch of the VM; and
launching a determined number of cloud functions (CF) to temporarily absorb the increased workload while the VM is being launched,
wherein the number of CFs that are launched provide an aggregate amount of memory equal to the VM,
wherein requests are routed to the CFs to begin immediate processing while the VM is being launched, and
wherein, when the VM is online, the CFs are terminated and new requests are shifted to the VM.

### Claim 10 (depends on 9)

The method of claim 9, wherein an application programming interface (API) is provided to communicate with the CFs, and wherein parameters of requests to the CFs are formatted in accordance with the API.

### Claim 11 (depends on 9)

The method of claim 9, wherein the resource scaling policy, when implemented, causes an initial pool of cloud functions (CFs) to be initiated, and wherein the CFs in the pool are in a suspended state until such time as an increased memory requirement of the application is detected.

### Claim 12 (depends on 11)

The method of claim 11, wherein each CF in the pool is initially loaded with a corresponding instance of the application.

### Claim 13 (depends on 9)

The method of claim 9, wherein the MRC changes over time as patterns in a workload of the application change.

### Claim 14 (depends on 13)

The method of claim 13, wherein the MRC is recomputed periodically, and wherein the MRC is generated based on a sampling of a subset of spatially hashed memory references.

### Claim 15

A method for reactively scaling resources for memory-intensive applications and for automatically adapting to workload changes with no pre-specified thresholds, said method comprising:
generating a miss ratio curve (MRC) of an application based on application runtime statistics that are extracted from the application;
modeling the application's MRC as a hyperbola;
identifying an area on the hyperbola where the MRC is determined to satisfy a flatten threshold;
establishing a resource allocation threshold based on the identified area on the hyperbola, the resource allocation threshold indicating how many resources are to be provisioned for the application; and
scaling the resources that are to be provisioned for the application using a resource scaling policy that is based on the resource allocation threshold,
wherein the resource scaling policy, when implemented, causes an initial pool of cloud functions (CFs) to be initiated, and
wherein the CFs in the pool are in a suspended state until such time as an increased memory requirement of the application is detected.

### Claim 16 (depends on 15)

The method of claim 15, wherein runtime measurements are used to estimate an average amount of memory a request of the application requires.