# US 2023/0155958 A1 · Method for optimal resource selection based on available gpu resource analysis in large-scale container platform

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0155958 A1
- Document ID: US20230155958A1
- Status: Published application
- Kind: A1
- Application number: 17/983,770
- Filed: 2022-11-09
- Published: 2023-05-18
- Earliest claimed: 2021-11-12
- Assignees: Korea Electronics Technology Institute
- Applicants: Korea Electronics Technology Institute
- Inventors: Jae Hoon AN; Young Hwan KIM
- Web page: https://patentgrep.com/patent/US20230155958A1

## CPC

- H04L 47/783: Distributed allocation of resources, e.g. bandwidth brokers
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

There are provided a method and an apparatus for cloud management, which selects optimal resources based on graphic processing unit (GPU) resource analysis in a large-scale container platform environment. According to an embodiment, a GPU bottleneck phenomenon occurring in an application of a large-scale container environment may be reduced by processing partitioned allocation of GPU resources, rather than existing 1:1 allocation, through real-time GPU data analysis (application of a threshold) and synthetic analysis of GPU performance degrading factors.

## Claims

### Claim 1

A cloud management method comprising:
collecting, by a cloud management device, metric data regarding a GPU use state in a large-scale container platform environment;
generating, by the cloud management device, a GPU map by analyzing the collected metric data; and
performing, by the cloud management device, a scheduling operation for distributed deployment of GPU resources, based on the generated GPU map.

### Claim 2 (depends on 1)

The cloud management method of claim 1, wherein performing the scheduling operation comprises:
detecting an overload pod performance of which is degraded due to a reduced speed or lack of resources, based on the generated GPU map;
recording information regarding the detected overload pod;
selecting an optimal candidate node for allocating all resources or sharing and allocating some of the resources of the overload pod, in replacement of the overload pod, based on the generated GPU map; and
performing the scheduling operation, based on the recorded information regarding the overload pod and information regarding the optimal candidate node.

### Claim 3 (depends on 2)

The cloud management method of claim 2, wherein, when selecting the optimal candidate node, the cloud management device is configured to select one or more first nodes to which requested GPU resources are allocable, to calculate an appropriate score with respect to the selected first node, and to select a most appropriate node based on each appropriate score.

### Claim 4 (depends on 3)

The cloud management method of claim 3, wherein the cloud management device is configured to detect occurrence of overload with respect to all pods registered at the GPU map periodically according to a pre-set first period, and, when a new pod is generated, to detect occurrence of overload with respect to the generated pod irrespective of whether the first period comes.

### Claim 5 (depends on 4)

The cloud management method of claim 4, wherein the cloud management device is configured to iteratively perform the scheduling operation of GPU resources, based on the generated GPU map, periodically according to a pre-set second period, and
wherein the second period is relatively longer than the first period, so that the scheduling operation of GPU resources is performed based on information on the overload pod that is accumulatively recorded.

### Claim 6 (depends on 3)

The cloud management method of claim 3, wherein, when selecting the first node, the cloud management device is configured to calculate a number of available GPUs for each node, and to predict a memory usage of each pod included in each node, based on the metric data, and to select the first node to which the requested GPU resources are allocable, based on the calculated number of available GPUs for each node and a result of predicting the memory usage of each pod.

### Claim 7 (depends on 3)

The cloud management method of claim 3, wherein, when selecting the optimal candidate node, the cloud management device is configured to select, as the optimal candidate node first, a first node included in a node having a smallest number of pods participating in allocation of GPU resources, among nodes to which a first node corresponding to a pre-set upper range of appropriate scores belongs.

### Claim 8 (depends on 5)

The cloud management method of claim 5, wherein the cloud management device is configured to select a second node having a history of failing to schedule, and to perform a rescheduling operation of GPU resources with respect to the selected second node periodically according to a pre-set third period.

### Claim 9 (depends on 2)

The cloud management method of claim 2, when, after the scheduling operation is performed, when there is a difference between a prediction value obtained before the scheduling operation and a real value obtained after the scheduling operation by comparing, the cloud management device is configured to add a weight to a prediction value of a next period of a pod in which the difference occurs, by a difference value.

### Claim 10

A computer-readable recording medium having a computer program recorded thereon to perform a cloud management method, the method comprising:
collecting, by a cloud management device, metric data regarding a GPU use state in a large-scale container platform environment;
generating, by the cloud management device, a GPU map by analyzing the collected metric data; and
performing, by the cloud management device, a scheduling operation for distributed deployment of GPU resources, based on the generated GPU map.

### Claim 11

A cloud management device comprising:
a communication unit configured to collect metric data regarding a GPU use state in a large-scale container platform environment; and
a processor configured to generate a GPU map by analyzing the collected metric data, and to perform a scheduling operation for distributed deployment of GPU resources, based on the generated GPU map.
