US 2022/0027746 A1 · GRADIENT-BASED AUTO-TUNING FOR MACHINE LEARNING AND DEEP LEARNING MODELS

Abstract

Herein, horizontally scalable techniques efficiently configure machine learning algorithms for optimal accuracy and without informed inputs. In an embodiment, for each particular hyperparameter, and for each epoch, a computer processes the particular hyperparameter. An epoch explores one hyperparameter based on hyperparameter tuples. A respective score is calculated from each tuple. The tuple contains a distinct combination of values, each of which is contained in a value range of a distinct hyperparameter. All values of a tuple that belong to the particular hyperparameter are distinct. All values of a tuple that belong to other hyperparameters are held constant. The value range of the particular hyperparameter is narrowed based on an intersection point of a first line based on the scores and a second line based on the scores. A machine learning algorithm is optimally configured from repeatedly narrowed value ranges of hyperparameters. The configured algorithm is invoked to obtain a result.

Claims20

Dependent claims

A method comprising:

for each iteration of a plurality of iterations:
selecting a plurality of values for a hyperparameter of a machine learning (ML) algorithm,
training the ML algorithm based on the plurality of values for the hyperparameter of the ML algorithm,
calculating a plurality of scores based on said training the ML algorithm,
identifying two lines based on the plurality of scores based on said training the ML algorithm, and
estimating, based on the two lines based on said plurality of scores, a best value for the hyperparameter of the ML algorithm, wherein the best value estimates one selected from the group consisting of:
a minimum duration of said training the ML algorithm and
a maximum fitness of the ML algorithm;
configuring the ML algorithm based on said best value of an iteration of the plurality of iterations;
invoking the ML algorithm to obtain a result.

7 more elements

One or more computer-readable non-transitory media storing instructions that, when executed by one or more processors, cause:

for each iteration of a plurality of iterations:
selecting a plurality of values for a hyperparameter of a machine learning (ML) algorithm,
training the ML algorithm based on the plurality of values for the hyperparameter of the ML algorithm,
calculating a plurality of scores based on said training the ML algorithm,
identifying two lines based on the plurality of scores based on said training the ML algorithm, and
estimating, based on the two lines based on said plurality of scores, a best value for the hyperparameter of the ML algorithm, wherein the best value estimates one selected from the group consisting of:
a minimum duration of said training the ML algorithm and
a maximum fitness of the ML algorithm;
configuring the ML algorithm based on said best value of an iteration of the plurality of iterations;
invoking the ML algorithm to obtain a result.

7 more elements

Dependencies are inferred from the claim text.

Gradient-based auto-tuning for machine learning and deep learning models

Life of this patent

Gradient-based auto-tuning for machine learning and deep learning models

Life of this patent

Abstract

Claims20