# US 2017/0256018 A1 · Graphics processing unit resource sharing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0256018 A1
- Document ID: US20170256018A1
- Status: Published application
- Kind: A1
- Application number: 15/059,822
- Filed: 2016-03-03
- Published: 2017-09-07
- Earliest claimed: 2016-03-03
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Anshul Gandhi; Hui Lei; Jayaram Kallapalayam Radhakrishnan; Charles O. Schulz; Shu Tao
- Web page: https://patentgrep.com/patent/US20170256018A1

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 1/60: Memory management
- G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Examples of GPU resource sharing among distributed applications in a distributed computing environment are disclosed. In one example, a method includes receiving a first request from a first distributed application of the plurality of distributed applications for first requested GPU resources. The method may further include receiving a second request from a second distributed application of the plurality of distributed applications for second requested GPU resources. The method may also include receiving response from each of the plurality of computing nodes indicating an availability of GPU resources for each of the plurality of computing nodes. Additionally, the method may include, responsive to determining that at least one of the first and second requests can be fulfilled by at least one of the plurality of computing nodes, allocating a first set of GPU slices for the first application and allocating a second set of GPU slices for the second application.

## Claims

### Claim 1

A computer-implemented method for graphics processing unit (GPU) resource sharing among a plurality of distributed applications in a distributed computing environment comprising a plurality of computing nodes, each of the plurality of computing nodes comprising a GPU, the method comprising:
receiving a first request from a first distributed application of the plurality of distributed applications for first requested GPU resources among the plurality of computing nodes, the GPU resources comprising a processor and a memory;
receiving a second request from a second distributed application of the plurality of distributed applications for second requested GPU resources among the plurality of computing nodes;
receiving a response from each of the plurality of computing nodes indicating an availability of GPU resources for each of the plurality of computing nodes; and
responsive to determining that at least one of the first request and the second request can be fulfilled by at least one of the plurality of computing nodes, allocating a first set of GPU slices for the first application and allocating a second set of GPU slices for the second application,
wherein allocating the first set of GPU slices for the first application and allocating the second set of GPU slices for the second slice is performed according to a fairness policy, wherein the fairness policy establishes that an allocated percent of the memory is less than or equal to an allocated percent of the processor.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
enabling the first application and the second application to execute concurrently within the first set of GPU slices and the second set of GPU slices respectively.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the first set of GPU slices is inaccessible to the second application and wherein the second set of GPU slices is inaccessible to the first application.

### Claim 4

(canceled)

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving a third request from a third distributed application of the plurality of distributed applications for third requested GPU resources among the plurality of computing nodes.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, further comprising:
responsive to determining that the third request can be fulfilled by at least one of the plurality of computing nodes, allocating a third set of GPU slices for the third application.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the processor comprises multiple GPU cores, wherein each GPU core of the multiple GPU cores comprises a plurality of hardware threads.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the first request is a request for at least one of a desired minimum and a desired maximum amount of resources, and wherein the second request is a request for at least one of a desired minimum and a desired maximum amount of resources.

### Claim 9

A system for graphics processing unit (GPU) resource sharing among a plurality of distributed applications in a distributed computing environment comprising a plurality of computing nodes, each of the plurality of computing nodes comprising a GPU, the system comprising:
a processor in communication with one or more types of memory, the processor configured to:
receive a first request from a first distributed application of the plurality of distributed applications for first requested GPU resources among the plurality of computing nodes, the GPU resources comprising a processor and a memory;
receive a second request from a second distributed application of the plurality of distributed applications for second requested GPU resources among the plurality of computing nodes;
receive a response from each of the plurality of computing nodes indicating an availability of GPU resources for each of the plurality of computing nodes; and
responsive to determining that at least one of the first request and the second request can be fulfilled by at least one of the plurality of computing nodes, allocate a first set of GPU slices for the first application and allocating a second set of GPU slices for the second application,
wherein allocating the first set of GPU slices for the first application and allocating the second set of GPU slices for the second slice is performed according to a fairness policy, wherein the fairness policy establishes that an allocated percent of the memory is less than or equal to an allocated percent of the processor.

### Claim 10 (depends on 9)

The system of claim 9, wherein the processor is further configured to:
enable the first application and the second application to execute concurrently within the first set of GPU slices and the second set of GPU slices respectively.

### Claim 11 (depends on 9)

The system of claim 9, wherein the first set of GPU slices is inaccessible to the second application and wherein the second set of GPU slices is inaccessible to the first application.

### Claim 12

(canceled)

### Claim 13 (depends on 9)

The system of claim 9, wherein the processor is further configured to:
receive a third request from a third distributed application of the plurality of distributed applications for third requested GPU resources among the plurality of computing nodes.

### Claim 14 (depends on 13)

The system of claim 13, wherein the processor is further configured to:
responsive to determining that the third request can be fulfilled by at least one of the plurality of computing nodes, allocate a third set of GPU slices for the third application.

### Claim 15 (depends on 9)

The system of claim 9, wherein the processor comprises multiple GPU cores, wherein each GPU core of the multiple GPU cores comprises a plurality of hardware threads.

### Claim 16 (depends on 9)

The system of claim 9, wherein the first request is a request for at least one of a desired minimum and a desired maximum amount of resources, and wherein the second request is a request for at least one of a desired minimum and a desired maximum amount of resources.

### Claim 17

A computer program product for graphics processing unit (GPU) resource sharing among a plurality of distributed applications in a distributed computing environment comprising a plurality of computing nodes, each of the plurality of computing nodes comprising a GPU, the computer program product comprising:
a non-transitory storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising:
receiving a first request from a first distributed application of the plurality of distributed applications for first requested GPU resources among the plurality of computing nodes, the GPU resources comprising a processor and a memory;
receiving a second request from a second distributed application of the plurality of distributed applications for second requested GPU resources among the plurality of computing nodes;
receiving a response from each of the plurality of computing nodes indicating an availability of GPU resources for each of the plurality of computing nodes; and
responsive to determining that at least one of the first request and the second request can be fulfilled by at least one of the plurality of computing nodes, allocating a first set of GPU slices for the first application and allocating a second set of GPU slices for the second application
wherein allocating the first set of GPU slices for the first application and allocating the second set of GPU slices for the second slice is performed according to a fairness policy, wherein the fairness policy establishes that an allocated percent of the memory is less than or equal to an allocated percent of the processor.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein the method further comprises:
enabling the first application and the second application to execute concurrently within the first set of GPU slices and the second set of GPU slices respectively.

### Claim 19 (depends on 17)

The computer program product of claim 17, wherein the first set of GPU slices is inaccessible to the second application and wherein the second set of GPU slices is inaccessible to the first application.

### Claim 20

(canceled)
