US 2019/0155660 A1 · SCHEDULING SYSTEM FOR COMPUTATIONAL WORK ON HETEROGENEOUS HARDWARE

Abstract

The technology includes methods, processes, and systems for virtualizing graphic processing unit (GPU) memory. Example embodiments of the technology include managing an amount of GPU memory used by one or more processes, such as Application Programming Interfaces (APIs), that directly or indirectly impact one or more other processes running on the same GPU. Managing and/or virtualizing the amount of GPU memory may ensure that an end user does not receive a GPU out-of-memory error because the API request is impacted by the processing of other API requests. A virtual machine with access to a GPU may be organized with one or more job slots that are configured to specify the number of processes that are able to run concurrently on a specific virtual machine. A process may be configured on each virtual machine running a software program or API and is used to schedule work based on GPU memory requirements.

Claims20

Dependent claims

A computer-implemented method, comprising:

under the control of one or more computer systems configured with executable instructions,
receiving a first request to process first data according to the first request;
determining an amount of GPU resources to process the first request;
determining original available GPU resources on a GPU operably interconnected with a virtual host;
associating the first request and the first data with the original available GPU resources being at least the amount of GPU resources to process the first request;
receiving a second request to process second data according to the second request;
determining an amount of GPU resources to process the second request;
determining remaining available GPU resources on the GPU, wherein the remaining GPU resources include at least the difference between the original available GPU resources and the amount of GPU resources to process the second request;
and
as a result of determining the remaining available GPU resources, associating the second request and the second data with the remaining available GPU resources.

7 more elements

Dependencies are inferred from the claim text.

Scheduling system for computational work on heterogeneous hardware

Life of this patent

Scheduling system for computational work on heterogeneous hardware

Life of this patent

Abstract

Claims20