# US 10,769,529 B2 · Controlled adaptive optimization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,529 B2 - Document ID: US10769529B2 - Status: Granted - Kind: B2 - Application number: 16/657,356 - Filed: 2019-10-18 - Published: 2020-06-04 - Granted: 2020-09-08 - Earliest claimed: 2018-12-04 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Sashank Jakkam Reddi; Sanjiv Kumar; Manzil Zaheer; Satyen Chandrakant Kale - Web page: https://patentgrep.com/patent/US10769529B2 ## CPC - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 20/00: Machine learning - G06N 3/045: Combinations of networks - G06N 3/09: Supervised learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive effective learning rate while also ensuring that the effective learning rate is non-increasing. ## Claims ### Claim 1 A computer-implemented method for optimizing machine-learned models that provides improved convergence properties, the method comprising: for each of a plurality of iterations: determining, by one or more computing devices, a gradient of a loss function that evaluates a performance of a machine-learned model that comprises a plurality of parameters; determining, by the one or more computing devices, a current learning rate control value based on the gradient of the loss function, wherein the current learning rate control value equals a most recent learning rate control value minus an update value, wherein a magnitude of the update value is a function of the gradient of the loss function but not the most recent learning rate control value, and wherein a polarity of the update value is a function of both the gradient of the loss function and the most recent learning rate control value; determining, by the one or more computing devices, a current effective learning rate based at least in part on the current learning rate control value; and determining, by the one or more computing devices, an updated set of values for the plurality of parameters of the machine-learned model based at least in part on the gradient of the loss function and according to the current effective learning rate; and providing, by the one or more computing devices, an optimized version of the machine-learned model as an output, the optimized version of the machine-learned model comprising a final set of values for the plurality of parameters; wherein, for at least one of the plurality of iterations, the polarity of the update value is positive such that the current learning rate control value is less than the most recent learning rate control value, whereby the current effective learning rate is greater than a most recent effective learning rate. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the update value is equal to a square of the gradient of the loss function multiplied by a sign function applied to the most recent learning rate control value minus the square of the gradient of the loss function and multiplied by a scaling coefficient that is equal to one minus an update scaling parameter. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein, over the plurality of iterations, the update scaling parameter is held constant. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein, over the plurality of iterations, the update scaling parameter is increased so as to provide increasing influence to past learning rate control values. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein determining, by the one or more computing devices, the updated set of values for the plurality of parameters of the machine-learned model based at least in part on the gradient of the loss function and according to the current effective learning rate comprises: updating, by the one or more computing devices, a current momentum value based at least in part on the gradient of the loss function and one or more previous momentum values respectively from one or more previous iterations; and determining, by the one or more computing devices, the updated set of values for the plurality of parameters of the machine-learned model based at least in part on the current momentum value and according to the current effective learning rate. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein determining, by the one or more computing devices, the current effective learning rate based at least in part on the current learning rate control value comprises dividing, by the one or more computing devices, a current learning rate by a square root of the current learning rate control value. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein determining, by the one or more computing devices, the current effective learning rate based at least in part on the current learning rate control value comprises dividing, by the one or more computing devices, a current learning rate by a square root of the current learning rate control value plus an adaptivity control value. ### Claim 8 A computing system, comprising: one or more processors; and one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising, for each of one or more iterations: determining a gradient of a loss function that evaluates a performance of a machine-learned model that comprises a plurality of parameters; determining a current learning rate control value based on the gradient of the loss function, wherein the current learning rate control value equals a most recent learning rate control value minus an update value, wherein a magnitude of the update value is equal to a square of the gradient of the loss function times a scaling coefficient, and wherein a polarity of the update value is a function of both the gradient of the loss function and the most recent learning rate control value; determining a current effective learning rate based at least in part on the current learning rate control value; and determining an updated set of values for the plurality of parameters of the machine-learned model based at least in part on the gradient of the loss function and according to the current effective learning rate. ### Claim 9 (depends on 8) The computing system of claim 8, wherein the update value is equal to the square of the gradient of the loss function multiplied by a sign function applied to the most recent learning rate control value minus the square of the gradient of the loss function and multiplied by the scaling coefficient, wherein the scaling coefficient is equal to one minus an update scaling parameter. ### Claim 10 (depends on 8) The computing system of claim 8, wherein the one or more iterations comprise a plurality of iterations and wherein, for at least one of the plurality of iterations, the polarity of the update value is positive such that the current learning rate control value is less than the most recent learning rate control value, whereby the current effective learning rate is greater than a most recent effective learning rate. ### Claim 11 (depends on 8) The computing system of claim 8 wherein, over the one or more iterations, the update scaling parameter is held constant. ### Claim 12 (depends on 8) The computing system of claim 8, wherein, over the one or more iterations, the update scaling parameter is increased so as to provide increasing influence to past learning rate control values. ### Claim 13 (depends on 8) The computing system of claim 8, wherein determining, by the one or more computing devices, the updated set of values for the plurality of parameters of the machine-learned model based at least in part on the gradient of the loss function and according to the current effective learning rate comprises: updating, by the one or more computing devices, a current momentum value based at least in part on the gradient of the loss function and one or more previous momentum values respectively from one or more previous iterations; and determining, by the one or more computing devices, the updated set of values for the plurality of parameters of the machine-learned model based at least in part on the current momentum value and according to the current effective learning rate. ### Claim 14 (depends on 8) The computing system of claim 8, wherein determining, by the one or more computing devices, the current effective learning rate based at least in part on the current learning rate control value comprises dividing, by the one or more computing devices, a current learning rate by a square root of the current learning rate control value. ### Claim 15 (depends on 8) The computing system of claim 8, wherein determining, by the one or more computing devices, the current effective learning rate based at least in part on the current learning rate control value comprises dividing, by the one or more computing devices, a current learning rate by a square root of the current learning rate control value plus an adaptivity control value. ### Claim 16 One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising: for each of a plurality of iterations: determining a gradient of a loss function that evaluates a performance of a machine-learned model that comprises a plurality of parameters; determining a current learning rate control value based on the gradient of the loss function, wherein the current learning rate control value equals a most recent learning rate control value minus an update value, wherein the update value is equal to a square of the gradient of the loss function multiplied by a sign function applied to the most recent learning rate control value minus the square of the gradient of the loss function and multiplied by a scaling coefficient that is equal to one minus an update scaling parameter; determining, by the one or more computing devices, a current effective learning rate based at least in part on the current learning rate control value; and updating at least one of the plurality of parameters of the machine-learned model based at least in part on the gradient of the loss function and according to a current effective learning rate that is a function of the current learning rate control value. ### Claim 17 (depends on 16) The one or more non-transitory computer-readable media of claim 16, wherein the current effective learning rate is inversely correlated to the current learning rate control value. ### Claim 18 (depends on 16) The one or more non-transitory computer-readable media of claim 16, wherein the one or more iterations comprise a plurality of iterations and wherein, for at least one of the plurality of iterations, the polarity of the update value is positive such that the current learning rate control value is less than the most recent learning rate control value, whereby the current effective learning rate is greater than a most recent effective learning rate. ### Claim 19 (depends on 16) The one or more non-transitory computer-readable media of claim 16, wherein, over the one or more iterations, the update scaling parameter is held constant or increased.