US 11,526,434 B1Granted

Network-level garbage collection in an on-demand code execution system

AssigneeAmazon Technologies, Inc.

InventorsMarc John Brooker, Andrew J. Lusk, Mikhail Danilov

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods are described for management of garbage collection processes in an on-demand code execution system. An on-demand code execution system may execute user-submitted code on virtual machine instances. As each virtual machine instance executes code, garbage (e.g., memory that is allocated and then not properly de-allocated) may accumulate and may reduce the computing resources that the virtual machine instance makes available for allocation to further code executions. A garbage collection schedule manager may be used to generate a system-wide garbage collection schedule and manage garbage collection processes in accordance with the schedule. The garbage collection schedule manager may schedule garbage collection processes at times when demand for computing resources is forecasted to be low, and may coordinate scheduling of garbage collection processes across multiple virtual machine instances and host computing devices to prevent garbage collection processes from impacting code execution performance.

Claims19

Dependent claims

A system for managing garbage collection in an on-demand code execution system, the system comprising:

a processor in communication with a non-transitory data store;
and
computer-executable instructions stored in the non-transitory data store, wherein the computer-executable instructions, when executed by the processor, configure the processor to perform operations including:
obtaining historical workload metrics regarding demand for a plurality of host computing devices, the demand comprising requests to execute user-submitted tasks on an on-demand code execution system;
determining, based at least in part on the historical workload metrics, a workload forecast for the plurality of host computing devices;
determining, based at least in part on the workload forecast, a garbage collection schedule, wherein the garbage collection schedule comprises a plurality of timeslots, wherein each of the plurality of timeslots is associated with a time at which the workload forecast predicts that future requests to execute user-submitted tasks will consume less than a threshold amount of computing resources of the plurality of host computing devices, and wherein a number of timeslots associated with individual times is determined based at least in part on the workload forecast;
receiving a request to schedule a first garbage collection process on a first virtual machine instance that is hosted on a host computing device of the plurality of host computing devices;
identifying a first open timeslot on the garbage collection schedule, the first open timeslot being associated with a first time;
and
assigning the first open timeslot to the first virtual machine instance, wherein assigning the first open timeslot to the first virtual machine instance causes the first virtual machine instance to execute the first garbage collection process at the first time.

7 more elements

A computer-implemented method comprising:

determining, based at least in part on historical workload metrics for a plurality of host computing devices that execute user-submitted tasks in an on-demand code execution system, a workload forecast;
determining, based at least in part on the workload forecast, a garbage collection schedule, wherein the garbage collection schedule comprises a plurality of timeslots, wherein each of the plurality of timeslots is associated with a time, and wherein a number of timeslots associated with individual times is determined based at least in part on the workload forecast;
receiving a request to schedule a first garbage collection process on a first virtual machine instance that is hosted on one of the plurality of host computing devices;
identifying a first open timeslot on the garbage collection schedule;
and
causing the first virtual machine instance to execute the first garbage collection process at a time associated with the first open timeslot.

3 more elements

Dependencies are inferred from the claim text.