# US 10,373,284 B2 · Capacity reservation for virtualized graphics processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,373,284 B2
- Document ID: US10373284B2
- Status: Granted
- Kind: B2
- Application number: 15/376,399
- Filed: 2016-12-12
- Published: 2018-06-14
- Granted: 2019-08-06
- Earliest claimed: 2016-12-12
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Douglas Cotton Kurtz; Malcolm Featonby; Umesh Chandani; Adithya Bhat; Yuxuan Liu; Mihir Sadruddin Surani
- Web page: https://patentgrep.com/patent/US10373284B2

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 9/455: Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F 9/5044: {considering hardware capabilities}
- G06F 9/505: {considering the load}
- G06F 9/5072: {Grid computing}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G09G 5/006: {Details of the interface to the display terminal (specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04)}
- G06T 2200/28: involving image processing hardware
- G09G 2370/022: Centralised management of display operation, e.g. in a server instead of locally
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

Methods, systems, and computer-readable media for capacity reservation for virtualized graphics processing are disclosed. A request is received to attach a virtual GPU to a virtual compute instance. The request comprises one or more constraints. Availability information is retrieved from a data store that indicates virtual GPUs available in a provider network and matching the one or more constraints. A virtual GPU is selected from among the available virtual GPUs in the availability information. The selected virtual GPU is reserved for attachment to the virtual compute instance. The virtual compute instance is implemented using CPU resources and memory resources of a physical compute instance, the virtual GPU is implemented using a physical GPU in the provider network, and the physical GPU is accessible to the physical compute instance over a network.

## Claims

### Claim 1

A system, comprising:
a data store comprising data indicative of virtual graphics processing units (GPUs) available in a multi-tenant provider network; and
one or more computing devices that implement an elastic graphics service, wherein the elastic graphics service is configured to:
receive a request to attach a virtual GPU to a virtual compute instance, wherein the virtual compute instance is implemented using central processing unit (CPU) resources and memory resources of a physical compute instance, wherein the virtual GPU is implemented using a physical GPU in the multi-tenant provider network, wherein the physical GPU is accessible to the physical compute instance over a network, and wherein the request indicates one or more constraints associated with selection of the virtual GPU; and
responsive to the received request:
retrieve a page from the data store, wherein the page indicates a plurality of available virtual GPUs matching the one or more constraints;
determine a selected virtual GPU from among the available virtual GPUs in the page;
store a reservation of the selected virtual GPU on behalf of the virtual compute instance; and
attach the selected virtual GPU to the virtual compute instance.

### Claim 2 (depends on 1)

The system as recited in claim 1, wherein the one or more constraints comprise a virtual GPU class constraint and a location constraint.

### Claim 3 (depends on 1)

The system as recited in claim 1, wherein the selected virtual GPU is determined randomly from among the available virtual GPUs in the page.

### Claim 4

A computer-implemented method, comprising:
receiving a request to attach a virtual graphics processing unit (GPU) to a virtual compute instance, wherein the request comprises one or more constraints; and
responsive to the received request:
retrieving, from a data store, data indicative of a plurality of virtual GPUs available in a provider network and matching the one or more constraints;
determining a selected virtual GPU from among the available virtual GPUs; and
reserving the selected virtual GPU for attachment to the virtual compute instance, wherein the selected virtual GPU is attached to the virtual compute instance, wherein the virtual compute instance is implemented using central processing unit (CPU) resources and memory resources of a physical compute instance, wherein the selected virtual GPU is implemented using a physical GPU in the provider network, and wherein the physical GPU is accessible to the physical compute instance over a network.

### Claim 5 (depends on 4)

The method as recited in claim 4, wherein the one or more constraints comprise an indication of a virtual GPU class of the virtual GPU.

### Claim 6 (depends on 4)

The method as recited in claim 4, wherein the one or more constraints comprise a network locality constraint associated with the virtual GPU.

### Claim 7 (depends on 4)

The method as recited in claim 4, wherein the one or more constraints comprise a geographical constraint associated with the virtual GPU.

### Claim 8 (depends on 4)

The method as recited in claim 4, wherein the selected virtual GPU is determined randomly from among the available virtual GPUs.

### Claim 9 (depends on 4)

The method as recited in claim 4, wherein the data store comprises a plurality of segments associated with the one or more constraints and a plurality of additional segments associated with one or more different constraints, wherein identifiers of the plurality of segments are placed in a queue in a random order, wherein the identifiers of one or more of the segments are removed from the queue based at least in part on a lack of available capacity, and wherein the data indicative of the plurality of virtual GPUs is retrieved from a first segment remaining in the queue.

### Claim 10 (depends on 4)

The method as recited in claim 4, wherein a page of query results is retrieved that comprises the data indicative of the plurality of virtual GPUs available in the provider network and matching the one or more constraints, and wherein the page is retrieved in a sequential order from among a plurality of pages listing available virtual GPUs matching the one or more constraints.

### Claim 11 (depends on 4)

The method as recited in claim 4, wherein a page of query results is retrieved that comprises the data indicative of the plurality of virtual GPUs available in the provider network and matching the one or more constraints, and wherein the page is retrieved in a random order from among a plurality of pages listing available virtual GPUs matching the one or more constraints.

### Claim 12 (depends on 4)

The method as recited in claim 4, further comprising:
failing to reserve one or more other virtual GPUs from among the available virtual GPUs prior to determining the selected virtual GPU from among the available virtual GPUs.

### Claim 13 (depends on 4)

The method as recited in claim 4, wherein a page of query results is retrieved that comprises the data indicative of the plurality of virtual GPUs available in the provider network and matching the one or more constraints, and wherein the page is retrieved from the data store after failing to reserve one or more other virtual GPUs in one or more other pages retrieved from the data store.

### Claim 14 (depends on 4)

The method as recited in claim 4, further comprising:
verifying a capacity of the provider network to meet the request prior to retrieving the data indicative of the plurality of virtual GPUs, wherein the capacity is verified using queries of a plurality of segments of the data store associated with the one or more constraints.

### Claim 15 (depends on 4)

The method as recited in claim 4, wherein the request indicates a plurality of requested virtual GPUs, and wherein the method further comprises:
failing to reserve one or more of the plurality of requested virtual GPUs; and
reverting a reservation of the selected virtual GPU.

### Claim 16 (depends on 4)

The method as recited in claim 4, further comprising:
initiating a sweeper process, wherein the sweeper process performs:
determining one or more reservations associated with one or more virtual GPUs that have not been utilized by one or more virtual compute instances indicated in the one or more reservations; and
modifying the data store to indicate an availability of the one or more virtual GPUs for new reservations.

### Claim 17

A non-transitory computer-readable storage medium storing program instructions computer-executable to perform:
receiving a request to attach a virtual graphics processing unit (GPU) to a virtual compute instance, wherein the virtual compute instance is implemented using central processing unit (CPU) resources and memory resources of a physical compute instance, wherein the virtual GPU is implemented using a physical GPU in the multi-tenant provider network, wherein the physical GPU is accessible to the physical compute instance over a network, and
wherein the request indicates one or more constraints associated with the virtual GPU; and
responsive to the received request:
retrieving availability information from a data store, wherein the data store comprises data indicative of virtual GPUs available in a multi-tenant provider network, and wherein the availability information indicates a plurality of available virtual GPUs matching the one or more constraints; determining a selected virtual GPU from among the available virtual GPUs in the availability information;
storing a reservation of the selected virtual GPU for the virtual compute instance; and
attaching the selected virtual GPU to the virtual compute instance.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium as recited in claim 17, wherein the one or more constraints comprise an indication of a virtual GPU class of the virtual GPU.

### Claim 19 (depends on 17)

The non-transitory computer-readable storage medium as recited in claim 17, wherein the one or more constraints comprise a network locality constraint associated with the virtual GPU or a geographical constraint associated with the virtual GPU.

### Claim 20 (depends on 17)

The non-transitory computer-readable storage medium as recited in claim 17, wherein the selected virtual GPU is determined randomly from among the available virtual GPUs in the availability information.