# US 2021/0241099 A1 · Meta cooperative training paradigms

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241099 A1
- Document ID: US20210241099A1
- Status: Published application
- Kind: A1
- Application number: 17/136,054
- Filed: 2020-12-29
- Published: 2021-08-05
- Earliest claimed: 2020-02-05
- Assignees: Baidu USA LLC
- Applicants: Baidu USA, LLC
- Inventors: Dingcheng LI; Haiyan YIN; Xu LI; Ping LI
- Web page: https://patentgrep.com/patent/US20210241099A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0475: Generative networks
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Generative adversarial models have several benefits; however, due to mode collapse, these generators face a quality-diversity trade-off (i.e., the generator models sacrifice generation diversity for increased generation quality). Presented herein are embodiments that improve the performance of adversarial content generation by decelerating mode collapse. In one or more embodiments, a cooperative training paradigm is employed where a second model is cooperatively trained with the generator and helps efficiently shape the data distribution of the generator against mode collapse. Moreover, embodiments of a meta learning mechanism may be used, where the cooperative update to the generator serves as a high-level meta task and which helps ensures the generator parameters after the adversarial update stay resistant against mode collapse. In experiments, tested employments demonstrated efficient slowdown of mode collapse for the adversarial text generators. Overall, embodiments outperformed the baseline approaches with significant margins in terms of both generation quality and diversity.

## Claims

### Claim 1

A computer-implemented method for training a generator comprising:
responsive to a stop condition having not been reached, performing steps comprising:
sampling a set of data points from a training data;
using a generator model, which comprises a set of generator parameter values, to generate a set of generated data points;
computing an adversarial loss for the generator model using an adversarial training loss function;
determining a set of intermediate generator parameter values for the generator model using the adversarial loss and gradient descent;
using the set of data points sampled from the training data as inputs into a second neural network model, which comprises a second neural network model set of parameter values, and into the generator model comprising the set of intermediate generator parameter values, computing a cooperative training loss for the generator model;
determining a meta gradient using the cooperate training loss;
updating the set of generator parameter values using an adversarial gradient, which is obtained using the adversarial loss for the generator model, and the meta gradient;
updating a set of discriminator parameter values for a discriminator model using an adversarial loss for the discriminator model; and
updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss for the second neural network model; and
responsive to the stop condition having been reached, outputting the generator model, which comprises a final updated set of generator parameter values.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1 further comprising as initial steps:
initializing at least the set of generator parameter values of the generator model and the set of discriminator parameter values of the discriminator model; and
pretraining the generator model using training data, the generator model, and the discriminator model.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2 wherein the second neural network model and the generator model share a same neural network structure and the method further comprises:
using at least some of the set of generator parameter values from the pretrained generator model as parameter values for the second neural network model.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1 wherein the step of updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss comprises:
using a maximum likelihood estimation (MLE) loss function.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4 wherein the step of updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss comprises:
minimizing Kullback-Leibler divergence between the second neural network model using the set of data points sampled from the training data and the second neural network model using a mixture of data points sampled from the training data and data points sampled from data points that were generated by the generator model.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5 wherein the mixture comprises an equal number or approximately equal number of data points from the training data and data points which were generated by the generator model.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1 wherein the adversarial loss for the discriminator model and the adversarial loss for the generator model are obtained by using a min-max loss function.

### Claim 8

A system comprising:
one or more processors; and
a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
responsive to a stop condition having not been reached, performing steps comprising:
sampling a set of data points from a training data having a first distribution;
using a generator model, which comprises a set of generator parameter values, to generate a set of generated data points;
computing an adversarial loss for the generator model using an adversarial training loss function;
determining a set of intermediate generator parameter values for the generator model using the adversarial loss and gradient descent;
using the set of data points sampled from the training data as inputs into a second neural network model, which comprises a second neural network model set of parameter values, and into the generator model comprising the set of intermediate generator parameter values, computing a cooperative training loss for the generator model;
determining a meta gradient using the cooperate training loss for the generator model;
updating the set of generator parameter values using an adversarial gradient, which is obtained using the adversarial loss for the generator model, and the meta gradient;
updating a set of discriminator parameter values for a discriminator model using an adversarial loss for the discriminator model; and
updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss for the second neural network model; and
responsive to the stop condition having been reached, outputting the generator model, which comprises a final updated set of generator parameter values.

### Claim 9 (depends on 8)

The system of claim 8 wherein the non-transitory computer-readable medium or media further comprises one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
initializing at least the set of generator parameter values of the generator model and the set of discriminator parameter values of the discriminator model; and
pretraining the generator model using training data, the generator model, and the discriminator model.

### Claim 10 (depends on 9)

The system of claim 9 wherein the second neural network model and the generator model share a same neural network structure and the non-transitory computer-readable medium or media further comprises one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
using at least some of the set of generator parameter values from the pretrained generator model as parameter values for the second neural network model.

### Claim 11 (depends on 8)

The system of claim 8 wherein the step of updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss comprises:
using a maximum likelihood estimation (MLE) loss function.

### Claim 12 (depends on 11)

The system of claim 11 wherein the step of updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss comprises:
minimizing Kullback-Leibler divergence between the second neural network model using the set of data points sampled from the training data and the second neural network model using a mixture of data points sampled from the training data and data points sampled from data points that were generated by the generator model.

### Claim 13 (depends on 12)

The system of claim 12 wherein the mixture comprises an equal number or approximately equal number of data points from the training data and data points which were generated by the generator model.

### Claim 14 (depends on 8)

The system of claim 8 wherein the adversarial loss for the discriminator model and the adversarial loss for the generator model are obtained by using a min-max loss function.

### Claim 15

A computer-implemented method for training a generator comprising:
responsive to a stop condition having not been reached, performing steps comprising:
generating a set of generated data points using a set of data points from a training dataset of real data and a generator model from a generative adversarial system comprising the generator model having a set of generator model parameter values and a discriminator model having a set of discriminator parameter values;
computing an adversarial loss for the generator model using an adversarial training loss function;
determining a set of intermediate generator parameter values for the generator model using the adversarial loss and gradient descent;
using the generator model with the set of intermediate generator parameter values and a second neural network model to cooperatively train the generator model to decelerate mode collapse of the generator model;
updating a set of discriminator parameter values for the discriminator model using an adversarial loss for the discriminator model; and
updating a set of parameter values for the second neural network model using a cooperative training loss for the second neural network model; and
responsive to the stop condition having been reached, outputting the generator model.

### Claim 16 (depends on 15)

The computer-implemented method of claim 15 wherein the step of using the generator model with the set of intermediate generator parameter values and a second neural network model to cooperatively train the generator model to decelerate mode collapse of the generator model comprises:
computing a cooperative training loss for the generator model using the set of data points sampled from the training dataset of real data as inputs into the second neural network model and into the generator model comprising the set of intermediate generator parameter values;
determining a meta gradient using the cooperate training loss for the generator model; and
updating the set of generator parameter values using an adversarial gradient, which is obtained using the adversarial loss for the generator model, and the meta gradient.

### Claim 17 (depends on 15)

The computer-implemented method of claim 15 further comprising as initial steps:
initializing at least the set of generator parameter values of the generator model and the set of discriminator parameter values of the discriminator model;
pretraining the generator model using training dataset and the generator model and the discriminator model; and
using at least some of the set of generator parameter values from the pretrained generator model as parameter values for the second neural network model.

### Claim 18 (depends on 15)

The computer-implemented method of claim 15 wherein the generator is a natural language text generator and the second neural network model is a language model.

### Claim 19 (depends on 15)

The computer-implemented method of claim 15 wherein the step of updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss comprises:
minimizing Kullback-Leibler divergence between the second neural network model using the set of data points sampled from the training dataset of real data and the second neural network model using a mixture of data points sampled from the training dataset of real data and data points sampled from data points that were generated by the generator model.

### Claim 20 (depends on 19)

The computer-implemented method of claim 19 wherein the mixture comprises an equal number or approximately equal number of data points from the training dataset and data points which were generated by the generator model.

### Claim 21

A system comprising:
one or more processors; and
a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
responsive to a stop condition having not been reached, performing steps comprising:
generating a set of generated data points using a set of data points from a training dataset of real data and a generator model from a generative adversarial system comprising the generator model having a set of generator model parameter values and a discriminator model having a set of discriminator parameter values;
computing an adversarial loss for the generator model using an adversarial training loss function;
determining a set of intermediate generator parameter values for the generator model using the adversarial loss and gradient descent;
using the generator model with the set of intermediate generator parameter values and a second neural network model to cooperatively train the generator model to decelerate mode collapse of the generator model;
updating a set of discriminator parameter values for the discriminator model using an adversarial loss for the discriminator model; and
updating a set of parameter values for the second neural network model using a cooperative training loss for the second neural network model; and
responsive to the stop condition having been reached, outputting the generator model.

### Claim 22 (depends on 21)

The system of claim 21 wherein the step of using the generator model with the set of intermediate generator parameter values and a second neural network model to cooperatively train the generator model to decelerate mode collapse of the generator model comprises:
computing a cooperative training loss for the generator model using the set of data points sampled from the training dataset of real data as inputs into the second neural network model and into the generator model comprising the set of intermediate generator parameter values;
determining a meta gradient using the cooperate training loss for the generator model; and
updating the set of generator parameter values using an adversarial gradient, which is obtained using the adversarial loss for the generator model, and the meta gradient.

### Claim 23 (depends on 21)

The system of claim 21 wherein the step of updating the second neural network model set of parameter values of the second neural network model using a cooperative training loss comprises:
minimizing Kullback-Leibler divergence between the second neural network model using the set of data points sampled from the training dataset of real data and the second neural network model using a mixture of data points sampled from the training dataset of real data and data points sampled from data points that were generated by the generator model.