# US 2023/0419122 A1 · Adaptive Learning Rates for Training Adversarial Models with Improved Computational Efficiency

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0419122 A1
- Document ID: US20230419122A1
- Status: Published application
- Kind: A1
- Application number: 18/341,397
- Filed: 2023-06-26
- Published: 2023-12-28
- Earliest claimed: 2022-06-24
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Hussein Hazimeh; Natalia Borisovna Ponomareva
- Web page: https://patentgrep.com/patent/US20230419122A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/094: Adversarial learning
- G06N 3/0475: Generative networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Provided are systems and methods that use a novel learning rate scheduling technique to dynamically adapt the learning rate of an adversarial model to maintain an appropriate balance between adversarial components of the model. The scheduling technique is driven by the fact that, in some settings, the loss of an ideal adversarial network can be analytically determined a priori. A scheduler component can thus operate to keep the loss of the optimized network close to that of an ideal adversarial net.

## Claims

### Claim 1

A computer-implemented method for training adversarial models with improved computational efficiency, the method comprising:
obtaining, by a computing system comprising one or more computing devices, one or more training samples;
processing, by the computing system, the one or more training samples with an adversarial machine learning model to generate one or more outputs, wherein the adversarial machine learning model comprises at least a first model component and a second model component that are adversarial to each other;
evaluating, by the computing system, a loss function based at least in part on the one or more outputs to determine a current loss value associated with the adversarial machine learning model;
determining, by the computing system, a distance between the current loss value associated with the adversarial machine learning model and an ideal loss value for the adversarial machine learning model;
determining, by the computing system, an adaptive learning rate value for at least one of the first model component and the second model component based at least in part on the distance between the current loss value associated with the adversarial machine learning model and the ideal loss value for the adversarial machine learning model; and
updating, by the computing system, the at least one of the first model component and the second model component according to the adaptive learning rate value.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein:
the loss function comprises a minimax function;
the first model component seeks to minimize the minimax function;
the second model component seeks to maximize the minimax function; and
the ideal loss value comprises a minimum value of the minimax function.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein:
the first model component is configured to generate a first output;
the second model component comprises a discriminator model configured to generate a second output comprising a probability that the first output belongs to a first distribution; and
the ideal loss value occurs when the probability output by the discriminator model is equal to one half.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein:
the adversarial machine learning model comprises a generative adversarial network;
the first model component comprises a generator network configured to generate the first output; and
the second model component comprises a discriminator network.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein the generator network is configured to generate a synthetic image.

### Claim 6 (depends on 3)

The computer-implemented method of claim 3, wherein
the adversarial machine learning model comprises a domain adversarial neural network;
the first model component comprises a feature extraction network configured to generate the first output comprising extracted features;
the second model component comprises a discriminator network; and
the domain adversarial neural network comprises a third model component configured to generate a task output based on the extracted features.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein:
determining, by the computing system, the adaptive learning rate value for the at least one of the first model component and the second model component comprises determining, by the computing system, the adaptive learning rate value for the second model component; and
updating, by the computing system, the at least one of the first model component and the second model component according to the adaptive learning rate value comprises updating, by the computing system, the second model component according to the adaptive learning rate value.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, further comprising:
updating, by the computing system, the first model component according to a fixed learning rate value.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the first model comprises an image synthesis model.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein determining, by the computing system, the adaptive learning rate value for at least one of the first model component and the second model component based at least in part on the distance between the current loss value associated with the adversarial machine learning model and the ideal loss value for the adversarial machine learning model comprises:
determining, by the computing system, a learning rate scaling value for the at least one of the first model component and the second model component based at least in part on the distance between the current loss value associated with the adversarial machine learning model and the ideal loss value for the adversarial machine learning model; and
scaling, by the computing system, a base learning rate value by the learning rate scaling value to obtain the adaptive learning rate value.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein:
when the current loss value is greater than the ideal loss value, the learning rate scaling value is greater than or equal to one;
when the current loss value is less than the ideal loss value, the learning rate scaling value is greater than zero and less than or equal to one.

### Claim 12 (depends on 10)

The computer-implemented method of claim 10, wherein determining, by the computing system, the learning rate scaling value comprises:
when the current loss value is greater than the ideal loss value, evaluating a first scheduling function with an argument of the distance between the current loss value associated with the adversarial machine learning model and the ideal loss value for the adversarial machine learning model; and
when the current loss value is less than the ideal loss value, evaluating a second scheduling function with an argument of the distance between the current loss value associated with the adversarial machine learning model and the ideal loss value for the adversarial machine learning model.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, wherein:
the first scheduling function comprises linear or exponential interpolation between one and a maximum value; and
the second scheduling function comprises linear or exponential interpolation between a minimum value and one.

### Claim 14 (depends on 1)

The computer-implemented method of claim 1, wherein the ideal loss value comprises the loss for the machine learning system when an output of first model component is indistinguishable, by the second model component, from a target distribution.

### Claim 15 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more training samples comprise a batch of a plurality of training samples.

### Claim 16 (depends on 1)

The computer-implemented method of claim 1, wherein the current loss value comprises an exponential moving average of a model loss over a number of batches.

### Claim 17

A computer system comprising:
one or more processors;
at least a first machine learning component,
wherein the first machine learning component was trained using the adaptive learning rate as described in any preceding claim, or
wherein the first machine learning component was jointly trained with a second machine learning component trained using the adaptive learning rate as described in any preceding claim; and
one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computer system to run at least the first machine learning component.

### Claim 18 (depends on 17)

The computer system of claim 17, wherein the first machine learning component was jointly trained with the second machine learning component using a loss function and the adaptive learning rate, wherein:
the loss function comprises a minimax function;
the first model component seeks to minimize the minimax function;
the second model component seeks to maximize the minimax function; and
the ideal loss value comprises a minimum value of the minimax function.

### Claim 19 (depends on 17)

The computer system of claim 17, wherein:
the first model component is configured to generate a first output;
the second model component comprises a discriminator model configured to generate a second output comprising a probability that the first output belongs to a first distribution; and
the ideal loss value occurs when the probability output by the discriminator model is equal to one half.

### Claim 20 (depends on 17)

The computer system of claim 17, wherein:
the adversarial machine learning model comprises a generative adversarial network;
the first model component comprises a generator network configured to generate the first output; and
the second model component comprises a discriminator network.
