# US 12,001,511 B2 · Systems and methods of resource configuration optimization for machine learning workloads

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,001,511 B2
- Document ID: US12001511B2
- Status: Granted
- Kind: B2
- Application number: 17/199,294
- Filed: 2021-03-11
- Published: 2022-09-15
- Granted: 2024-06-04
- Earliest claimed: 2021-03-11
- Assignees: Hewlett Packard Enterprise Development LP
- Applicants: Hewlett Packard Enterprise Development LP
- Inventors: Lianjie Cao; Faraz Ahmed; Puneet Sharma; Ali Tariq
- Web page: https://patentgrep.com/patent/US12001511B2

## CPC

- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 40/172: {Classification, e.g. identification}
- G06F 9/5016: {the resource being the memory}
- G06F 9/5022: {Mechanisms to release resources}
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 9/505: {considering the load}
- G06F 9/5061: {Partitioning or combining of resources}
- G06F 9/5066: {Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451)}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 9/5083: {Techniques for rebalancing the load in a distributed system}
- G06F 11/3051: {Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs (verification or detection of system hardware configuration G06F11/2247)}
- G06F 11/3414: {Workload generation, e.g. scripts, playback}
- G06F 18/24155: {Bayesian classification}
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N 20/00: Machine learning
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods can be configured to determine a plurality of computing resource configurations used to perform machine learning model training jobs. A computing resource configuration can comprise: a first tuple including numbers of worker nodes and parameter server nodes, and a second tuple including resource allocations for the worker nodes and parameter server nodes. At least one machine learning training job can be executed using a first computing resource configuration having a first set of values associated with the first tuple. During the executing the machine learning training job: resource usage of the worker nodes and parameter server nodes caused by a second set of values associated with the second tuple can be monitored, and whether to adjust the second set of values can be determined. Whether a stopping criterion is satisfied can be determined. One of the plurality of computing resource configurations can be selected.

## Claims

### Claim 1

A computer-implemented method comprising:
determining a plurality of computing resource configurations used to perform machine learning model training jobs, wherein a computing resource configuration comprises:
a first tuple including numbers of worker nodes and parameter server nodes, and
a second tuple including resource allocations for the worker nodes and parameter server nodes;
executing at least one machine learning training job using a first computing resource configuration having a first set of values associated with the first tuple, wherein during the executing the machine learning training job:
monitors resource usage of the worker nodes and parameter server nodes caused by a second set of values associated with the second tuple; and
determines whether to adjust the second set of values;
determining whether a stopping criterion is satisfied; and
selecting one of the plurality of computing resource configurations.

### Claim 2 (depends on 1)

The method of claim 1, wherein the executing the at least one machine learning training job comprises:
selecting the first tuple from a search space of tuples that each include different numbers of worker nodes and parameter server nodes;
executing a first loop with the first tuple to determine at least one performance metric associated with the first tuple;
selecting a different tuple from the search space of tuples;
executing the first loop with the different tuple to determine at least one performance metric associated with the different tuple; and
based on the at least one performance metrics associated with the first tuple and the different tuple, configuring the first loop to execute the next machine learning model training job with numbers of worker nodes and parameter server nodes specified in the different tuple.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
receiving a maximum number of nodes to use for the machine learning training job;
based on the maximum number of nodes, determine different combinations of a first number of worker nodes and a second number of parameter servers, wherein the first number and the second number in a combination of the different combinations are, together, less than or equal to the maximum number of nodes; and
determining a subset of the search space that only contains the different combinations.

### Claim 4 (depends on 3)

The method of claim 3, wherein the different tuple is selected using Bayesian optimization, the method further comprises:
assigning values to tuples not in the subset of the search space, wherein the values cause the tuples to be excluded from selection during execution of the first loop.

### Claim 5 (depends on 1)

The method of claim 1, wherein the second tuple includes at least one of a specified central processing unit (CPU) allocation or a memory allocation.

### Claim 6 (depends on 5)

The method of claim 5, wherein the executing the at least one machine learning training job comprises:
executing a second loop with the second tuple to determine at least one performance metric associated with the second tuple;
determining that a condition for re-allocating at least one of the specified CPU allocation and the memory allocation is satisfied;
reclaiming at least some resource from at least one of a CPU associated with the specified CPU allocation or a memory associated with the memory allocation; and
providing at least one of a new CPU allocation or a new memory allocation for execution of the at least one machine learning training job.

### Claim 7 (depends on 6)

The method of claim 6, wherein the condition is associated with at least one of a threshold level of resource utilization or a threshold level of resource idleness.

### Claim 8 (depends on 1)

The method of claim 1, wherein the stopping criterion comprises at least one of:
a number of computing resource configurations to be tested; or
an improvement of at least one performance metric between the first computing resource configuration and a second computing resource configuration, wherein the improvement is compared against a specified threshold level of improvement.

### Claim 9 (depends on 1)

The method of claim 1, wherein the optimal configuration is used in execution of remaining machine learning jobs of the at least one machine learning job.

### Claim 10 (depends on 1)

The method of claim 1, wherein:
the at least one machine learning job is a hyperparameter tuning machine learning job,
the at least one machine learning job is executed on a container comprising the worker nodes and the parameter nodes, and
the container is part of a distributed training environment.

### Claim 11

A system comprising:
at least one processor; and
a memory storing instructions that, when executed by the at least one processor, cause the system to perform a method comprising:
determining a plurality of computing resource configurations used to perform machine learning model training jobs, wherein a computing resource configuration comprises:
a first tuple including numbers of worker nodes and parameter server nodes, and
a second tuple including resource allocations for the worker nodes and parameter server nodes;
executing at least one machine learning training job using a first computing resource configuration having a first set of values associated with the first tuple, wherein during the executing the machine learning training job:
monitors resource usage of the worker nodes and parameter server nodes caused by a second set of values associated with the second tuple; and
determines whether to adjust the second set of values;
determining whether a stopping criterion is satisfied; and
selecting one of the plurality of computing resource configurations.

### Claim 12 (depends on 11)

The system of claim 11, wherein the executing the at least one machine learning training job comprises:
selecting the first tuple from a search space of tuples that each include different numbers of worker nodes and parameter server nodes;
executing a first loop with the first tuple to determine at least one performance metric associated with the first tuple;
selecting a different tuple from the search space of tuples;
executing the first loop with the different tuple to determine at least one performance metric associated with the different tuple; and
based on the at least one performance metrics associated with the first tuple and the different tuple, configuring the first loop to execute the next machine learning model training job with numbers of worker nodes and parameter server nodes specified in the different tuple.

### Claim 13 (depends on 12)

The system of claim 12, wherein the instruction cause the system to further perform:
receiving a maximum number of nodes to use for the machine learning training job;
based on the maximum number of nodes, determine different combinations of a first number of worker nodes and a second number of parameter servers, wherein the first number and the second number in a combination of the different combinations are, together, less than or equal to the maximum number of nodes; and
determining a subset of the search space that only contains the different combinations.

### Claim 14 (depends on 13)

The system of claim 13, wherein the different tuple is selected using Bayesian optimization, wherein the instructions cause the system to further perform:
assigning values to tuples not in the subset of the search space, wherein the values cause the tuples to be excluded from selection during execution of the first loop.

### Claim 15 (depends on 1)

The system of claim 1, wherein the second tuple includes at least one of a specified central processing unit (CPU) allocation or a memory allocation.

### Claim 16

A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform:
determining a plurality of computing resource configurations used to perform machine learning model training jobs, wherein a computing resource configuration comprises:
a first tuple including numbers of worker nodes and parameter server nodes, and
a second tuple including resource allocations for the worker nodes and parameter server nodes;
executing at least one machine learning training job using a first computing resource configuration having a first set of values associated with the first tuple, wherein during the executing the machine learning training job:
monitors resource usage of the worker nodes and parameter server nodes caused by a second set of values associated with the second tuple; and
determines whether to adjust the second set of values;
determining whether a stopping criterion is satisfied; and
selecting one of the plurality of computing resource configurations.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein the executing the at least one machine learning training job comprises:
selecting the first tuple from a search space of tuples that each include different numbers of worker nodes and parameter server nodes;
executing a first loop with the first tuple to determine at least one performance metric associated with the first tuple;
selecting a different tuple from the search space of tuples;
executing the first loop with the different tuple to determine at least one performance metric associated with the different tuple; and
based on the at least one performance metrics associated with the first tuple and the different tuple, configuring the first loop to execute the next machine learning model training job with numbers of worker nodes and parameter server nodes specified in the different tuple.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the instruction cause the computing system to further perform:
receiving a maximum number of nodes to use for the machine learning training job;
based on the maximum number of nodes, determine different combinations of a first number of worker nodes and a second number of parameter servers, wherein the first number and the second number in a combination of the different combinations are, together, less than or equal to the maximum number of nodes; and
determining a subset of the search space that only contains the different combinations.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, wherein the different tuple is selected using Bayesian optimization, wherein the instructions cause the computing system to further perform:
assigning values to tuples not in the subset of the search space, wherein the values cause the tuples to be excluded from selection during execution of the first loop.

### Claim 20 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein the second tuple includes at least one of a specified central processing unit (CPU) allocation or a memory allocation.