# US 2021/0241164 A1 · Distributed hyperparameter tuning and load balancing for mathematical models > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241164 A1 - Document ID: US20210241164A1 - Status: Published application - Kind: A1 - Application number: 16/778,587 - Filed: 2020-01-31 - Published: 2021-08-05 - Earliest claimed: 2020-01-31 - Assignees: salesforce.com, inc. - Applicants: salesforce.com, Inc. - Inventors: Bradford William Powley; Noah William Burbank; Rowan Cassius - Web page: https://patentgrep.com/patent/US20210241164A1 ## CPC - G06N 3/08: Learning methods - G06N 20/00: Machine learning - G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - H04L 67/1001: for accessing one among a plurality of replicated servers - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Methods, systems, and devices for distributed hyperparameter tuning and load balancing are described. A device (e.g., an application server) may generate a first set of combinations of hyperparameter values associated with training a mathematical model. The mathematical model may include a machine learning model, an optimization model, or any combination. The device may identify a subset of combinations from the first set of combinations that are associated with a computational runtime that exceeds a first threshold and may distribute the subset of combinations across a set of machines. The device may then test each of the first set of combinations in a parallel processing operation to generate a first set of validation error values and may test a second set of combinations of hyperparameter values using an objective function that is based on the first set of validation error values. ## Claims ### Claim 1 A method for data processing, comprising: generating a first plurality of combinations of hyperparameter values associated with training a mathematical model; identifying a subset of combinations from the first plurality of combinations that are associated with a computational runtime that exceeds a first threshold; distributing the subset of combinations across a plurality of machines such that each machine of the plurality of machines is assigned a number of the subset of combinations that is less than a second threshold; testing each of the first plurality of combinations against the mathematical model using the plurality of machines in a parallel processing operation to generate a first plurality of validation error values; and testing a second plurality of combinations of hyperparameter values against the mathematical model using an objective function that is based at least in part on the first plurality of validation error values. ### Claim 2 (depends on 1) The method of claim 1, wherein testing the second plurality of combinations of hyperparameter values comprises: running the objective function using a serial processing operation. ### Claim 3 (depends on 1) The method of claim 1, further comprising: identifying the second plurality of combinations of hyperparameter values based at least in part on the objective function. ### Claim 4 (depends on 1) The method of claim 1, further comprising: initializing the objective function based at least in part on the first plurality of validation error values. ### Claim 5 (depends on 1) The method of claim 1, wherein identifying the subset of combinations from the first plurality of combinations comprises: identifying one or more hyperparameter values associated with a hyperparameter, wherein the one or more hyperparameter values are associated with the computational runtime that exceeds the first threshold. ### Claim 6 (depends on 5) The method of claim 5, wherein the first threshold is based at least in part on a prior experience associated with the one or more hyperparameter values, an inference from previous runs, a known benchmark, or a combination thereof. ### Claim 7 (depends on 1) The method of claim 1, further comprising: distributing a remaining subset of combinations of hyperparameter values from the first plurality of combinations of hyperparameter values across the plurality of machines based at least in part on a random distribution. ### Claim 8 (depends on 1) The method of claim 1, further comprising: tuning the first threshold based at least in part on a number of the plurality of machines. ### Claim 9 (depends on 1) The method of claim 1, wherein the objective function comprises a Bayesian optimization function. ### Claim 10 (depends on 1) The method of claim 1, wherein the computational runtime is based at least in part on a runtime for one of the plurality of machines. ### Claim 11 (depends on 1) The method of claim 1, wherein the first plurality of combinations of hyperparameter values and the second plurality of combinations of hyperparameter values are different. ### Claim 12 (depends on 1) The method of claim 1, wherein the first plurality of combinations of hyperparameter values are defined by a user and comprise one or more hyperparameter values associated with a plurality of hyperparameters. ### Claim 13 (depends on 1) The method of claim 1, wherein the plurality of machines comprises servers, virtual machines, bare metal machines, a plurality of threads on a single multi-thread processor, or a combination thereof. ### Claim 14 (depends on 1) The method of claim 1, wherein the mathematical model comprises a machine learning model, an optimization model, or a combination thereof. ### Claim 15 An apparatus for data processing, comprising: a processor, memory coupled with the processor; and instructions stored in the memory and executable by the processor to cause the apparatus to: generate a first plurality of combinations of hyperparameter values associated with training a mathematical model; identify a subset of combinations from the first plurality of combinations that are associated with a computational runtime that exceeds a first threshold; distribute the subset of combinations across a plurality of machines such that each machine of the plurality of machines is assigned a number of the subset of combinations that is less than a second threshold; test each of the first plurality of combinations against the mathematical model using the plurality of machines in a parallel processing operation to generate a first plurality of validation error values; and test a second plurality of combinations of hyperparameter values against the mathematical model using an objective function that is based at least in part on the first plurality of validation error values. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the instructions to test the second plurality of combinations of hyperparameter values are executable by the processor to cause the apparatus to: run the objective function using a serial processing operation. ### Claim 17 (depends on 15) The apparatus of claim 15, wherein the instructions are further executable by the processor to cause the apparatus to: identify the second plurality of combinations of hyperparameter values based at least in part on the objective function. ### Claim 18 (depends on 15) The apparatus of claim 15, wherein the instructions are further executable by the processor to cause the apparatus to: initialize the objective function based at least in part on the first plurality of validation error values. ### Claim 19 (depends on 15) The apparatus of claim 15, wherein the instructions to identify the subset of combinations from the first plurality of combinations are executable by the processor to cause the apparatus to: identify one or more hyperparameter values associated with a hyperparameter, wherein the one or more hyperparameter values are associated with the computational runtime that exceeds the first threshold. ### Claim 20 A non-transitory computer-readable medium storing code for data processing, the code comprising instructions executable by a processor to: generate a first plurality of combinations of hyperparameter values associated with training a mathematical model; identify a subset of combinations from the first plurality of combinations that are associated with a computational runtime that exceeds a first threshold; distribute the subset of combinations across a plurality of machines such that each machine of the plurality of machines is assigned a number of the subset of combinations that is less than a second threshold; test each of the first plurality of combinations against the mathematical model using the plurality of machines in a parallel processing operation to generate a first plurality of validation error values; and test a second plurality of combinations of hyperparameter values against the mathematical model using an objective function that is based at least in part on the first plurality of validation error values.