# US 2020/0394748 A1 · Placement optimization for virtualized graphics processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394748 A1
- Document ID: US20200394748A1
- Status: Published application
- Kind: A1
- Application number: 16/914,090
- Filed: 2020-06-26
- Published: 2020-12-17
- Earliest claimed: 2015-11-11
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Nicholas Patrick Wilt; Ashutosh Tambe
- Web page: https://patentgrep.com/patent/US20200394748A1

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 9/445: Program loading or initiating (bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57)
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 9/50: Allocation of resources, e.g. of the central processing unit [CPU]
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 2009/45562: {Creating, deleting, cloning virtual machine instances}
- G06F 2009/4557: {Distribution of virtual machine instances; Migration and load balancing}
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

Methods, systems, and computer-readable media for placement optimization for virtualized graphics processing are disclosed. A provider network comprises a plurality of instance locations for physical compute instances and a plurality of graphics processing unit (GPU) locations for physical GPUs. A GPU location for a physical GPU or an instance location for a physical compute instance is selected in the provider network. The GPU location or instance location is selected based at least in part on one or more placement criteria. A virtual compute instance with attached virtual GPU is provisioned. The virtual compute instance is implemented using the physical compute instance in the instance location, and the virtual GPU is implemented using the physical GPU in the GPU location. The physical GPU is accessible to the physical compute instance over a network. An application is executed using the virtual GPU on the virtual compute instance.

## Claims

### Claim at position 1

-20. (canceled)

### Claim 21

A system, comprising:
one or more processors and one or more memories to store computer-executable instructions that, when executed, cause the one or more processors to:
attach a virtual graphics processing unit (GPU) to one or more containers on a compute instance, wherein the compute instance is implemented using central processing unit (CPU) resources and memory resources, and wherein the virtual GPU is implemented using a physical GPU accessible to the compute instance over a network; and
execute an application in the one or more containers on the compute instance, wherein execution of the application generates one or more GPU instructions that are sent from the compute instance to the virtual GPU over the network and executed using the virtual GPU.

### Claim 22 (depends on 21)

The system as recited in claim 21, wherein the one or more memories store additional computer-executable instructions that, when executed, cause the one or more processors to:
configure one or more GPU parameters of the one or more containers.

### Claim 23 (depends on 22)

The system as recited in claim 22, wherein the one or more GPU parameters comprise one or more GPU types specified for the execution of the application.

### Claim 24 (depends on 21)

The system as recited in claim 21, wherein the one or more memories store additional computer-executable instructions that, when executed, cause the one or more processors to:
configuring one or more parameters of the one or more containers, wherein the one or more parameters indicate an application programming interface (API) shim associated with the application, wherein the API shim intercepts calls from the application to a graphics API.

### Claim 25 (depends on 21)

The system as recited in claim 21, wherein the compute instance comprises a node in a node pool.

### Claim 26 (depends on 21)

The system as recited in claim 21, wherein the compute instance comprises a virtual machine (VM), wherein the VM is implemented using the CPU resources and the memory resources of a physical compute instance.

### Claim 27

A computer-implemented method, comprising:
attaching a virtual graphics processing unit (GPU) to one or more containers on a compute instance, wherein the compute instance is implemented using central processing unit (CPU) resources and memory resources, and wherein the virtual GPU is implemented using a physical GPU accessible to the compute instance over a network; and
executing an application in the one or more containers on the compute instance, wherein execution of the application generates one or more GPU instructions that are sent from the compute instance to the virtual GPU over the network and executed using the virtual GPU.

### Claim 28 (depends on 27)

The method as recited in claim 27, further comprising:
configuring one or more GPU parameters of the one or more containers.

### Claim 29 (depends on 28)

The method as recited in claim 28, wherein the one or more GPU parameters comprise one or more GPU types recommended for the execution of the application.

### Claim 30 (depends on 27)

The method as recited in claim 27, further comprising:
configuring one or more parameters of the one or more containers, wherein the one or more parameters indicate an application programming interface (API) shim, wherein the API shim intercepts calls from the application to a graphics API.

### Claim 31 (depends on 27)

The method as recited in claim 27, wherein the compute instance comprises a node in a computing cluster.

### Claim 32 (depends on 27)

The method as recited in claim 27, wherein the compute instance comprises a virtual machine (VM), wherein the VM is implemented using the CPU resources and the memory resources of a physical compute instance.

### Claim 33 (depends on 32)

The method as recited in claim 32, wherein the VM is hosted by a cloud provider network, wherein the cloud provider network comprises a plurality of instance locations for physical compute instances and a plurality of GPU locations for physical GPUs, and wherein one of the instance locations is selected for the physical compute instance or one of the GPU locations is selected for the physical GPU based at least in part on one or more placement criteria for optimization of placement.

### Claim 34

One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more processors, perform:
attaching a virtual graphics processing unit (GPU) to one or more containers on a compute instance, wherein the compute instance is implemented using central processing unit (CPU) resources and memory resources, and wherein the virtual GPU is implemented using a physical GPU accessible to the compute instance over a network; and
executing an application in the one or more containers on the compute instance, wherein execution of the application generates one or more GPU instructions that are sent from the compute instance to the virtual GPU over the network and executed using the virtual GPU.

### Claim 35 (depends on 34)

The one or more non-transitory computer-readable storage media as recited in claim 34, further comprising additional program instructions that, when executed on or across the one or more processors, perform:
configuring one or more GPU parameters of the one or more containers.

### Claim 36 (depends on 35)

The one or more non-transitory computer-readable storage media as recited in claim 35, wherein the one or more GPU parameters comprise one or more GPU classes permissible for the execution of the application.

### Claim 37 (depends on 34)

The one or more non-transitory computer-readable storage media as recited in claim 34, further comprising additional program instructions that, when executed on or across the one or more processors, perform:
configuring one or more parameters of the one or more containers, wherein the one or more parameters indicate an application programming interface (API) shim associated with the application, wherein the API shim intercepts calls from the application to a graphics API.

### Claim 38 (depends on 34)

The one or more non-transitory computer-readable storage media as recited in claim 34, wherein the compute instance comprises a node in a cluster.

### Claim 39 (depends on 34)

The one or more non-transitory computer-readable storage media as recited in claim 34, wherein the compute instance comprises a virtual machine (VM), wherein the VM is implemented using the CPU resources and the memory resources of a physical compute instance.

### Claim 40 (depends on 34)

The one or more non-transitory computer-readable storage media as recited in claim 34, wherein the VM is hosted by a multi-tenant provider network, wherein the multi-tenant provider network comprises a plurality of instance locations for physical compute instances and a plurality of GPU locations for physical GPUs, and wherein one of the instance locations is selected for the physical compute instance or one of the GPU locations is selected for the physical GPU based at least in part on one or more placement criteria for optimization of placement.
