# US 11,907,675 B2 · Generating training datasets for training neural networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,675 B2 - Document ID: US11907675B2 - Status: Granted - Kind: B2 - Application number: 16/746,674 - Filed: 2020-01-17 - Published: 2020-07-23 - Granted: 2024-02-20 - Earliest claimed: 2019-01-18 - Assignees: Uber Technologies, Inc. - Applicants: Uber Technologies, Inc. - Inventors: Felipe Petroski Such; Aditya Rawal; Joel Anthony Lehman; Kenneth Owen Stanley; Jeffrey Michael Clune - Web page: https://patentgrep.com/patent/US11907675B2 ## CPC - G06F 40/42: Data-driven translation - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06F 40/58: Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0475: Generative networks - G06N 3/0495: Quantised networks; Sparse networks; Compressed networks - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N 3/09: Supervised learning - G06N 3/092: Reinforcement learning - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 10/776: Validation; Performance evaluation - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06N 3/045: Combinations of networks - G06N 3/047: Probabilistic or stochastic networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A generative cooperative network (GCN) comprises a dataset generator model and a learner model. The dataset generator model generates training datasets used to train the learner model. The trained learner model is evaluated according to a reference training dataset. The dataset generator model is modified according to the evaluation. The training datasets, the dataset generator model, and the leaner model are stored by the GCN. The trained learner model is configured to receive input and to generate output based on the input. ## Claims ### Claim 1 A computer implemented method comprising: receiving a reference training dataset including one or more reference training samples; accessing a training dataset generator model configured to generate training datasets; accessing a learner model configured to process the generated training datasets and the reference training dataset; in one or more iterations: generating, by the training dataset generator model, a training dataset according to noise as input, the training dataset including one or more training samples generated by the training dataset generator model; training the learner model using the generated training dataset by determining a first loss based on a difference between an output value predicted by the learner model and a corresponding output value generated by the training dataset generator model, wherein parameters of the learner model are adjusted based on the first loss; determining a second loss based on execution of the trained learner model using the reference training dataset; and adjusting parameters of the training dataset generator model based on the second loss determined based on execution of the trained learner model using the reference training dataset; storing one or more training datasets generated by the training dataset generator model; and storing the training dataset generator model with adjusted parameters and the trained learner model. ### Claim 2 (depends on 1) The computer implemented method of claim 1, wherein adjusting parameters of the training dataset generator model comprises: determining a gradient of loss of the learner model on the reference training dataset with respect to parameters of the training dataset generator model; and modifying the parameters of the training dataset generator model based on the determined gradient. ### Claim 3 (depends on 1) The computer implemented method of claim 1, wherein the learner model is a supervised learning based model and evaluating the trained learner model using the reference training dataset comprises evaluating a softmax function using an output value predicted by the learner model. ### Claim 4 (depends on 1) The computer implemented method of claim 1, wherein the learner model is an unsupervised learning based model and evaluating the trained learner model using the reference training dataset comprises determining a reconstruction loss value based on a reconstructed output predicted by the learner model and the input of the learner model. ### Claim 5 (depends on 1) The computer implemented method of claim 1, wherein the learner model is a reinforcement learning based model and evaluating the trained learner model using the reference training dataset comprises determining a policy gradient. ### Claim 6 (depends on 1) The computer implemented method of claim 1, wherein the training dataset generator model is a deconvolution network configured to generate an image as a training sample. ### Claim 7 (depends on 1) The computer implemented method of claim 1, wherein an agent navigates a moveable apparatus according to output generated by the learner model. ### Claim 8 (depends on 7) The computer implemented method of claim 7, wherein the moveable apparatus is an autonomous vehicle driving in a real-world environment and the learner model receives sensor data of the autonomous vehicle. ### Claim 9 (depends on 7) The computer implemented method of claim 7, wherein the moveable apparatus is a robot navigating through an obstacle course and the learner model receives sensor data of the robot. ### Claim 10 (depends on 1) The computer implemented method of claim 1, wherein the learner model is configured to receive an image as input and to output an object identified in the image. ### Claim 11 (depends on 1) The computer implemented method of claim 1, wherein the learner model is configured to receive one or more sentences specified in a first language and to output a translation of the one or more sentences in a second language. ### Claim 12 A computer system comprising: one or more processors; and a non-transitory computer readable storage medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving a reference training dataset including one or more reference training samples; accessing a training dataset generator model configured to generate training datasets; accessing a learner model configured to process the generated training datasets and the reference training dataset; in one or more iterations: generating, by the training dataset generator model, a training dataset according to noise as input, the training dataset including one or more training samples generated by the training dataset generator model; training the learner model using the generated training dataset by determining a first loss based on a difference between an output value predicted by the learner model and a corresponding output value generated by the training dataset generator model, wherein parameters of the learner model are adjusted based on the first loss; determining a second loss based on execution of the trained learner model using the reference training dataset; and adjusting parameters of the training dataset generator model based on the second loss determined based on execution of the trained learner model using the reference training dataset; storing one or more training datasets generated by the training dataset generator model; and storing the training dataset generator model with adjusted parameters and the trained learner model. ### Claim 13 (depends on 12) The computer system of claim 12, wherein adjusting parameters of the training dataset generator model comprises: determining a gradient of loss of the learner model on the reference training dataset with respect to parameters of the training dataset generator model; and modifying the parameters of the training dataset generator model based on the determined gradient. ### Claim 14 (depends on 12) The computer system of claim 12, wherein the learner model is a supervised learning based model and evaluating the trained learner model using the reference training dataset comprises evaluating a softmax function using an output value predicted by the learner model. ### Claim 15 A non-transitory computer readable storage medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving a reference training dataset including one or more reference training samples; accessing a training dataset generator model configured to generate training datasets; accessing a learner model configured to process the generated training datasets and the reference training dataset; in one or more iterations: generating, by the training dataset generator model, a training dataset according to noise as input, the training dataset including one or more training samples generated by the training dataset generator model; training the learner model using the generated training dataset by determining a first loss based on a difference between an output value predicted by the learner model and a corresponding output value generated by the training dataset generator model, wherein parameters of the learner model are adjusted based on the first loss; determining a second loss based on execution of the trained learner model using the reference training dataset; and adjusting parameters of the training dataset generator model based on the second loss determined based on execution of the trained learner model using the reference training dataset; storing one or more training datasets generated by the training dataset generator model; and storing the training dataset generator model with adjusted parameters and the trained learner model. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein adjusting parameters of the training dataset generator model comprises: determining a gradient of loss of the learner model on the reference training dataset with respect to parameters of the training dataset generator model; and modifying the parameters of the training dataset generator model based on the determined gradient. ### Claim 17 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the learner model is a supervised learning based model and evaluating the trained learner model using the reference training dataset comprises evaluating a softmax function using an output value predicted by the learner model.