# US 2022/0035672 A1 · Resource allocation for tuning hyperparameters of large-scale deep learning workloads

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0035672 A1
- Document ID: US20220035672A1
- Status: Published application
- Kind: A1
- Application number: 16/945,037
- Filed: 2020-07-31
- Published: 2022-02-03
- Earliest claimed: 2020-07-31
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Saritha Vinod; Anto Ajay Raj John; Asis Kumar Patra
- Web page: https://patentgrep.com/patent/US20220035672A1

## CPC

- G06F 9/5016: {the resource being the memory}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 9/5044: {considering hardware capabilities}
- G06F 9/505: {considering the load}
- G06N 3/086: using evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 2209/505: Clust
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method, computer system, and a computer program product may perform hyperparameter tuning. Hyperparameter search input data may be received. A search algorithm to generate a set of hyperparameters may be invoked. A hyperparameter tuning resource allocator may receive physical resource data regarding a cluster. The hyperparameter tuning resource allocator may determine a required workload and may use a received number of parallel experiments to determine the required workload. The hyperparameter tuning resource allocator may determine whether an option to mandate the workload batch size was selected. The cluster may perform the number of parallel experiments to search and tune the set of hyperparameters. The hyperparameter tuning resource allocator may manage the cluster according to the physical resource data and the required workload and by using tensor swapping.

## Claims

### Claim 1

A computer-implemented method performed by one or more processors, the method comprising:
receiving hyperparameter search input data, the hyperparameter search input data comprising a number of parallel experiments and an option to mandate a workload batch size for a hyperparameter search;
invoking a search algorithm to generate a set of hyperparameters;
receiving, at a hyperparameter tuning resource allocator, physical resource data regarding a cluster;
determining, via the hyperparameter tuning resource allocator, a required workload, wherein the hyperparameter tuning resource allocator uses the received number of parallel experiments to determine the required workload;
determining, via the hyperparameter tuning resource allocator, whether the option to mandate the workload batch size was selected; and
performing, via the cluster, the number of parallel experiments to search the set of hyperparameters and to tune the set of hyperparameters, wherein the hyperparameter tuning resource allocator manages the cluster, during the performing, according to the physical resource data and the required workload and by using tensor swapping.

### Claim 2 (depends on 1)

The method of claim 1, wherein the cluster comprises at least one accelerator and at least one central processing unit;
wherein the at least one accelerator comprises an accelerator memory;
wherein the at least one central processing unit comprises a CPU memory; and
wherein the physical resource data regarding the cluster relates to the at least one accelerator, to the at least one central processing unit, to the accelerator memory, and to the CPU memory.

### Claim 3 (depends on 1)

The method of claim 1, wherein in response to determining that the option to mandate the workload batch size was selected, the hyperparameter tuning resource allocator creates and manages virtual accelerators within the at least one accelerator.

### Claim 4 (depends on 1)

The method of claim 1, wherein the hyperparameter tuning resource allocator uses a training controller to launch the performing of the number of parallel experiments on the cluster.

### Claim 5 (depends on 2)

The method of claim 2, wherein the hyperparameter tuning resource allocator manages floating point operations per second of the at least one accelerator.

### Claim 6 (depends on 1)

The method of claim 1, wherein in response to determining that the option to mandate the workload batch size was not selected, the hyperparameter tuning resource allocator dynamically scales a batch size of the parallel experiments based on the required workload.

### Claim 7 (depends on 1)

The method of claim 1, wherein the cluster comprises at least one accelerator and at least one central processing unit;
wherein the tensor swapping comprises transferring at least one tensor from the at least one accelerator to the at least one central processing unit and comprises the at least one central processing unit returning the at least one tensor to the at least one accelerator; and
wherein the at least one tensor is generated via the at least one accelerator.

### Claim 8

A computer system for hyperparameter tuning, comprising:
one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage media, and program instructions stored on at least one of the one or more computer-readable tangible storage media for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, wherein the computer system is capable of performing a method comprising:
receiving hyperparameter search input data, the hyperparameter search input data comprising a number of parallel experiments and an option to mandate a workload batch size for a hyperparameter search;
invoking a search algorithm to generate a set of hyperparameters;
receiving, at a hyperparameter tuning resource allocator, physical resource data regarding a cluster;
determining, via the hyperparameter tuning resource allocator, a required workload, wherein the hyperparameter tuning resource allocator uses the received number of parallel experiments to determine the required workload;
determining, via the hyperparameter tuning resource allocator, whether the option to mandate the workload batch size was selected; and
performing, via the cluster, the number of parallel experiments to search the set of hyperparameters and to tune the set of hyperparameters, wherein the hyperparameter tuning resource allocator manages the cluster, during the performing, according to the physical resource data and the required workload and by using tensor swapping.

### Claim 9 (depends on 8)

The computer system of claim 8, wherein the cluster comprises at least one accelerator and at least one central processing unit;
wherein the at least one accelerator comprises an accelerator memory;
wherein the at least one central processing unit comprises a CPU memory; and
wherein the physical resource data regarding the cluster relates to the at least one accelerator, to the at least one central processing unit, to the accelerator memory, and to the CPU memory.

### Claim 10 (depends on 8)

The computer system of claim 8, wherein in response to determining that the option to mandate the workload batch size was selected, the hyperparameter tuning resource allocator creates and manages virtual accelerators within the at least one accelerator.

### Claim 11 (depends on 8)

The computer system of claim 8, wherein the hyperparameter tuning resource allocator uses a training controller to launch the performing of the number of parallel experiments on the cluster.

### Claim 12 (depends on 9)

The computer system of claim 9, wherein the hyperparameter tuning resource allocator manages floating point operations per second of the at least one accelerator.

### Claim 13 (depends on 8)

The computer system of claim 8, wherein in response to determining that the option to mandate the workload batch size was not selected, the hyperparameter tuning resource allocator dynamically scales a batch size of the parallel experiments depending on the required workload.

### Claim 14 (depends on 8)

The computer system of claim 8, wherein the cluster comprises at least one accelerator and at least one central processing unit;
wherein the tensor swapping comprises transferring at least one tensor from the at least one accelerator to the at least one central processing unit and comprises the at least one central processing unit returning the at least one tensor to the at least one accelerator; and
wherein the at least one tensor is generated via the at least one accelerator.

### Claim 15

A computer program product for hyperparameter tuning, comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions being executable by a processor to cause the processor to perform a method comprising:
receiving hyperparameter search input data, the hyperparameter search input data comprising a number of parallel experiments and an option to mandate a workload batch size for a hyperparameter search;
invoking a search algorithm to generate a set of hyperparameters;
receiving, at a hyperparameter tuning resource allocator, physical resource data regarding a cluster;
determining, via the hyperparameter tuning resource allocator, a required workload, wherein the hyperparameter tuning resource allocator uses the received number of parallel experiments to determine the required workload;
determining, via the hyperparameter tuning resource allocator, whether the option to mandate the workload batch size was selected; and
performing, via the cluster, the number of parallel experiments to search the set of hyperparameters and to tune the set of hyperparameters, wherein the hyperparameter tuning resource allocator manages the cluster, during the performing, according to the physical resource data and the required workload and by using tensor swapping.

### Claim 16 (depends on 15)

The computer program product of claim 15, wherein the cluster comprises at least one accelerator and at least one central processing unit;
wherein the at least one accelerator comprises an accelerator memory;
wherein the at least one central processing unit comprises a CPU memory; and
wherein the physical resource data regarding the cluster relates to the at least one accelerator, to the at least one central processing unit, to the accelerator memory, and to the CPU memory.

### Claim 17 (depends on 15)

The computer program product of claim 15, wherein in response to determining that the option to mandate the workload batch size was selected, the hyperparameter tuning resource allocator creates and manages virtual accelerators within the at least one accelerator.

### Claim 18 (depends on 15)

The computer program product of claim 15, wherein the hyperparameter tuning resource allocator uses a training controller to launch the performing of the number of parallel experiments on the cluster.

### Claim 19 (depends on 16)

The computer program product of claim 16, wherein the hyperparameter tuning resource allocator manages floating point operations per second of the at least one accelerator.

### Claim 20 (depends on 15)

The computer program product of claim 15, wherein in response to determining that the option to mandate the workload batch size was not selected, the hyperparameter tuning resource allocator dynamically scales a batch size of the parallel experiments depending on the required workload.
