# US 2022/0277229 A1 · Design learning: learning design policies based on interactions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0277229 A1
- Document ID: US20220277229A1
- Status: Published application
- Kind: A1
- Application number: 17/696,445
- Filed: 2022-03-16
- Published: 2022-09-01
- Earliest claimed: 2019-01-21
- Assignees: Capital One Services, LLC
- Applicants: Capital One Services, LLC
- Inventors: Omar FLOREZ CHOQUE; Anish KHAZANE; Alan SALIMOV
- Web page: https://patentgrep.com/patent/US20220277229A1

## CPC

- G06N 3/08: Learning methods
- G06N 20/00: Machine learning
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/045: Combinations of networks
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems, methods, and articles of manufacture for learning design policies based on user interactions. One example includes determining a first task for an environment, receiving data from a plurality of data sources, determining a first time step associated with the received data, determining a plurality of candidate actions for the determined first time step, computing a respective probability value of each candidate action achieving the first task at the first time step based on a first machine learning (ML) model, determining that a first candidate action has a greater probability value for achieving the first task at the first time step relative to the remaining plurality of candidate actions, determining that the first candidate action has not been implemented in the environment at the first time step, and generating an indication specifying to implement the first candidate action as part of a policy to achieve the first task.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving a first task, of a plurality of tasks, for an application, wherein the first task comprises receiving a review of the application;
extracting a plurality of features from data received from a plurality of data sources;
determining a plurality of candidate actions for a first time interval;
computing, based on a first machine learning (ML) model applied to the data and the plurality of features, a respective first probability value of each candidate action achieving the first task at the first time interval;
determining that a first candidate action of the plurality of candidate actions has a greater first probability value for achieving the first task at the first time interval relative to the first probability values of the remaining plurality of candidate actions;
generating, based on the first probability value of the first candidate action, an indication specifying to implement the first candidate action at the first time interval as part of a policy to achieve the first task;
generating, by a second ML model and based on the first ML model, the first candidate action, the data from the plurality of data sources, and the plurality of features, a natural language narrative describing the first candidate action;
computing, based on the first ML model, a review probability value reflecting whether a user will submit the review of the application; and
updating training data used to train the first ML model based on the review probability value.

### Claim 2 (depends on 1)

The method of claim 1, the training data labeled to specify that the corresponding training data achieves each of the plurality of tasks or the training data does not achieve each of the plurality of tasks.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
determining that the first candidate action has not been implemented at the first time interval prior to generating the indication specifying to implement the first candidate action at the first time as part of the policy to achieve the first task.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
computing, based on the first ML model, a second probability value for the first candidate action, the second probability value reflecting a likelihood that the first candidate action will occur at the first time interval.

### Claim 5 (depends on 4)

The method of claim 4, wherein the natural language narrative further describes the second probability value for the first candidate action, wherein the natural language narrative comprises a plurality of graphical symbols associated with a plurality of natural language concepts of the natural language narrative.

### Claim 6 (depends on 5)

The method of claim 5, the plurality of features comprising: sentiment data, objects identified in image data, natural language concepts extracted from audio data, and natural language concepts extracted from text data.

### Claim 7 (depends on 5)

The method of claim 5, further comprising:
receiving the data from the plurality of data sources, wherein the received data comprises audio data, image data, application data, and text data, wherein the plurality of natural language concepts of the natural language narrative comprise at least one natural language concept extracted from the audio data, at least one natural language concept extracted from the image data, and at least one natural language concept extracted from the text data.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
retraining the first ML model based on the updated training data.

### Claim 9

A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a processor, cause the processor to:
receive a first task, of a plurality of tasks, for an application, wherein the first task comprises receiving a review of the application;
extract a plurality of features from data received from a plurality of data sources;
determine a plurality of candidate actions for a first time interval;
compute, based on a first machine learning (ML) model applied to the data and the plurality of features, a respective first probability value of each candidate action achieving the first task at the first time interval;
determine that a first candidate action of the plurality of candidate actions has a greater first probability value for achieving the first task at the first time interval relative to the first probability values of the remaining plurality of candidate actions;
generate, based on the first probability value of the first candidate action, an indication specifying to implement the first candidate action at the first time interval as part of a policy to achieve the first task;
generate, by a second ML model and based on the first ML model, the first candidate action, the data from the plurality of data sources, and the plurality of features, a natural language narrative describing the first candidate action;
compute, based on the first ML model, a review probability value reflecting whether a user will submit the review of the application; and
update training data used to train the first ML model based on the review probability value.

### Claim 10 (depends on 9)

The computer-readable storage medium of claim 9, the training data labeled to specify that the corresponding training data achieves each of the plurality of tasks or the training data does not achieve each of the plurality of tasks.

### Claim 11 (depends on 9)

The computer-readable storage medium of claim 9, wherein the instructions further cause the processor to:
determine that the first candidate action has not been implemented at the first time interval prior to generating the indication specifying to implement the first candidate action at the first time interval as part of the policy to achieve the first task.

### Claim 12 (depends on 9)

The computer-readable storage medium of claim 9, wherein the instructions further cause the processor to:
compute, based on the first ML model, a second probability value for the first candidate action, the second probability value reflecting a likelihood that the first candidate action will occur at the first time interval.

### Claim 13 (depends on 12)

The computer-readable storage medium of claim 12, wherein the natural language narrative further describes the second probability value for the first candidate action, wherein the natural language narrative comprises a plurality of graphical symbols associated with a plurality of natural language concepts of the natural language narrative.

### Claim 14 (depends on 13)

The computer-readable storage medium of claim 13, the plurality of features comprising: sentiment data, objects identified in image data, natural language concepts extracted from audio data, and natural language concepts extracted from text data.

### Claim 15 (depends on 13)

The computer-readable storage medium of claim 13, wherein the instructions further cause the processor to:
receive the data from the plurality of data sources, wherein the received data comprises audio data, image data, application data, and text data, wherein the plurality of natural language concepts of the natural language narrative comprise at least one natural language concept extracted from the audio data, at least one natural language concept extracted from the image data, and at least one natural language concept extracted from the text data.

### Claim 16

A computing apparatus comprising:
a processor; and
a memory storing instructions that, when executed by the processor, cause the processor to:
receive a first task, of a plurality of tasks, for an application, wherein the first task comprises receiving a review of the application;
extract a plurality of features from data received from a plurality of data sources;
determine a plurality of candidate actions for a first time interval;
compute, based on a first machine learning (ML) model applied to the data and the plurality of features, a respective first probability value of each candidate action achieving the first task at the first time interval;
determine that a first candidate action of the plurality of candidate actions has a greater first probability value for achieving the first task at the first time interval relative to the first probability values of the remaining plurality of candidate actions;
generate, based on the first probability value of the first candidate action, an indication specifying to implement the first candidate action at the first time interval as part of a policy to achieve the first task;
generate, by a second ML model and based on the first ML model, the first candidate action, the data from the plurality of data sources, and the plurality of features, a natural language narrative describing the first candidate action;
compute, based on the first ML model, a review probability value reflecting whether a user will submit the review of the application; and
update training data used to train the first ML model based on the review probability value.

### Claim 17 (depends on 16)

The computing apparatus of claim 16, the training data labeled to specify that the corresponding training data achieves each of the plurality of tasks or the training data does not achieve each of the plurality of tasks.

### Claim 18 (depends on 16)

The computing apparatus of claim 16, wherein the instructions further cause the processor to:
determine that the first candidate action has not been implemented at the first time interval prior to generating the indication specifying to implement the first candidate action at the first time interval as part of the policy to achieve the first task.

### Claim 19 (depends on 16)

The computing apparatus of claim 16, wherein the instructions further cause the processor to:
compute, based on the first ML model, a second probability value for the first candidate action, the second probability value reflecting a likelihood that the first candidate action will occur at the first time interval.

### Claim 20 (depends on 19)

The computing apparatus of claim 19, wherein the natural language narrative further describes the second probability value for the first candidate action, wherein the natural language narrative comprises a plurality of graphical symbols associated with a plurality of natural language concepts of the natural language narrative.
