# US 2023/0267307 A1 · Systems and Methods for Generation of Machine-Learned Multitask Models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267307 A1
- Document ID: US20230267307A1
- Status: Published application
- Kind: A1
- Application number: 18/014,314
- Filed: 2020-07-23
- Published: 2023-08-24
- Earliest claimed: 2020-07-23
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Qifei Wang; Junjie Ke; Grace Chu; Gabriel Mintzer Bender; Luciano Sbaiz; Feng Yang; Andrew Gerald Howard; Alec Michael Go; Jeffrey M. Gilbert; Peyman Milanfar; Joshua William Charles Greaves
- Web page: https://patentgrep.com/patent/US20230267307A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods of the present disclosure are directed to a method for generating a machine-learned multitask model configured to perform tasks. The method can include obtaining a machine-learned multitask search model comprising candidate nodes. The method can include obtaining tasks and machine-learned task controller models associated with the tasks. As an example, for a task, the method can include using the task controller model to route a subset of the candidate nodes in a machine-learned task submodel for the corresponding task. The method can include inputting task input data to the task submodel to obtain a task output. The method can include generating, using the task output, a feedback value based on an objective function. The method can include adjusting parameters of the task controller model based on the feedback value.

## Claims

### Claim 1

A computer-implemented method for generating a machine-learned multitask model configured to perform a plurality of tasks, the method comprising:
obtaining, by one or more computing devices, a machine-learned multitask search model comprising a plurality of candidate nodes;
obtaining, by the one or more computing devices, the plurality of tasks and one or more machine-learned task controller models associated with the plurality of tasks;
for each task of the plurality of tasks:
using, by the one or more computing devices, the machine-learned task controller model respectively associated with the task to generate a routing that specifies a subset of the plurality of candidate nodes of the machine-learned multitask search model for inclusion in a machine-learned task submodel for the corresponding task;
inputting, by the one or more computing devices, task input data associated with the task to the corresponding machine-learned task submodel to obtain a task output;
generating, by the one or more computing devices using the task output, a feedback value based on an objective function; and
adjusting, by the one or more computing devices, one or more parameters of the respectively associated machine-learned task controller model based at least in part on the feedback value.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the method further comprises generating, by the one or more computing devices, the machine-learned multitask model, wherein the machine-learned multitask model comprises a combination of at least a subset of machine-learned task submodels of the plurality of machine-learned task submodels.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the method further comprises:
inputting, by the one or more computing devices, multitask training data associated with a machine-learned task submodel of the at least the subset of machine-learned task submodels to the machine-learned multitask model to obtain a multitask training output; and
adjusting, by the one or more computing devices, one or more parameters of the machine-learned multitask model based at least in part on the multitask training output.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein:
the feedback value comprises a reward value; and
the objective function comprises a reinforcement learning reward function.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein adjusting, by the one or more computing devices, the one or more parameters of the respectively associated machine-learned task controller model based at least in part on the feedback value comprises backpropagating the objective function through the corresponding machine-learned task submodel to reach the respectively associated machine-learned task controller model.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein:
for each task of the plurality of tasks:
inputting, by the one or more computing devices, the task input data associated with the task to the corresponding machine-learned task submodel to obtain the task output further comprises inputting, by the one or more computing devices, training data associated with the task to the corresponding machine-learned task submodel to obtain a training output;
generating, by the one or more computing devices using the task output, the feedback value based on the objective function further comprises generating, by the one or more computing devices using the training output, a loss value based on a task loss function; and
the method further comprises adjusting, by the one or more computing devices, the one or more parameters of at least one candidate node of the machine-learned multitask search model based on the plurality of loss values respectively associated with the plurality of tasks.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein:
the task input data comprises the training data; and
the task output comprises the training output.

### Claim 8 (depends on 6)

The computer-implemented method of claim 6, wherein the task input data comprises image data, and the task output comprises at least one of:
image classification data;
image recognition data;
object recognition data corresponding to one or more objects depicted in the image data; and
object segmentation data.

### Claim 9 (depends on 6)

The computer-implemented method of claim 6, wherein:
a respective task weight is associated with each task of the plurality of tasks; and
at least the objective function is configured to evaluate the task weight associated with the respective task.

### Claim 10 (depends on 6)

The computer-implemented method of claim 6, wherein:
a first loss value of the plurality of loss values is greater than a second loss value of the plurality of loss values; and
the one or more parameters of the at least one candidate node are adjusted based on the plurality of loss values and an adaptive loss function, wherein the adaptive loss function is configured to evaluate at least the difference between the first loss value and the second loss value.

### Claim 11 (depends on 1)

The computer-implemented method of claim 1, wherein:
the one or more machine-learned task controller models comprise a plurality of task controller models respectively associated with the plurality of tasks;
a first machine-learned task controller model associated with a first task is used to generate a first routing that specifies a first subset of the plurality of the candidate nodes;
a second machine-learned task controller model associated with a second task is used to generate a second routing that specifies a second subset of the plurality of the candidate nodes; and
the first subset of the plurality of candidate nodes and the second subset of the plurality of candidate nodes contain at least one shared candidate node.

### Claim 12 (depends on 1)

The computer-implemented method of claim 1, wherein, for each task of the plurality of tasks, the one or more parameters of the respectively associated machine-learned task controller model are adjusted based at least in part on an evaluation of a loss function.

### Claim 13 (depends on 1)

The computer-implemented method of claim 1, wherein at least one of the plurality of tasks comprises:
an image generation task;
a sound signal description task, wherein the task output of the sound signal description task comprises data describing a sound signal;
a text translation task, wherein the task output of the text translation task comprises a translation of text in a first natural language to a second natural language; or
a control data generation task, wherein the task output of the control data generation task comprises control data for controlling an agent which operates in a real-world environment.

### Claim 14

A computing system, comprising:
a machine-learned multitask model configured to generate a plurality of outputs for a respectively associated plurality of tasks, wherein the machine-learned multitask model comprises a plurality of nodes, wherein each node of the plurality of nodes is included in the machine-learned multitask model based at least in part on their inclusion in one or more of a plurality of machine-learned task submodels respectively associated with the plurality of tasks;
one or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising:
obtaining first task input data associated with a first task of the plurality of tasks;
obtaining second task input data associated with a second task of the plurality of tasks, the second task being different and distinct from the first task;
inputting the first task input data to the machine-learned multitask model to obtain a first task output that corresponds to the first task; and
inputting the second task input data to the machine-learned multitask model to obtain a second task output that corresponds to the second task.

### Claim 15 (depends on 14)

The computing system of claim 14, wherein:
the first task input data and the second task input data comprises image data;
the first task output comprises image classification data; and
the second task output comprises object recognition data corresponding to one or more objects depicted in the image data.

### Claim 16 (depends on 14)

The computing system of claim 14, wherein each node of the plurality of nodes of the machine-learned multitask model is selected for inclusion in the one or more of the plurality of machine-learned task submodels by one or more associated machine-learned task controller models.

### Claim 17 (depends on 14)

The computing system of claim 14, wherein:
the machine-learned multitask model comprises one or more neural networks; and
each of the plurality of nodes comprises at least one of:
one or more neurons; or
one or more functions.

### Claim 18 (depends on 14)

The computing system of claim 14, wherein:
the first task input data is processed by at least a first node of the machine-learned multitask model; and
the second task input data is processed by the first node of the machine-learned multitask model.

### Claim 19

One or more tangible, non-transitory computer readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:
obtaining a machine-learned multitask model configured to generate a plurality of outputs for a respectively associated plurality of tasks, wherein the machine-learned multitask model comprises a plurality of nodes, wherein each node of the plurality of nodes is included in the machine-learned multitask model based at least in part on their inclusion in one or more of a plurality of machine-learned task submodels respectively associated with the plurality of tasks;
obtaining first task input data associated with a first task of the plurality of tasks;
obtaining second task input data associated with a second task of the plurality of tasks, the second task being different and distinct from the first task;
inputting the first task input data to the machine-learned multitask model to obtain a first task output that corresponds to the first task; and
inputting the second task input data to the machine-learned multitask model to obtain a second task output that corresponds to the second task.

### Claim 20 (depends on 19)

The one or more tangible, non-transitory computer readable media of claim 19, wherein:
the first task input data and the second task input data comprises image data;
the first task output comprises image classification data; and
the second task output comprises object recognition data corresponding to one or more objects depicted in the image data; and
wherein each node of the plurality of nodes of the machine-learned multitask model is selected for inclusion in the one or more of the plurality of machine-learned task submodels by one or more associated machine-learned task controller models.

### Claim 21

(canceled)
