# US 2021/0280195 A1 · Infrastructure automation platform to assist in performing actions in response to tasks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0280195 A1
- Document ID: US20210280195A1
- Status: Published application
- Kind: A1
- Application number: 16/809,078
- Filed: 2020-03-04
- Published: 2021-09-09
- Earliest claimed: 2020-03-04
- Assignees: Accenture Global Solutions Limited
- Applicants: Accenture Global Solutions Limited
- Inventors: Madhan Kumar SRINIVASAN; Sumanta KAYAL; Moushom BORAH; Abhijit GHOSH
- Web page: https://patentgrep.com/patent/US20210280195A1

## CPC

- G06N 20/20: Ensemble learning
- G10L 17/00: Speaker identification or verification techniques
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 16/2379: {Updates performed during online database operations; commit processing}
- G06F 21/57: Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 5/04: Inference or reasoning models
- G06N 20/00: Machine learning
- G06V 40/172: {Classification, e.g. identification}
- G06N 5/045: Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A device may receive user personalized data and user activity data identifying tasks and actions performed by a user, and may perform natural language processing on the user personalized data and the user activity data to generate processed textual data. The device may train machine learning models based on the processed textual data to generate trained machine learning models, and may receive, from a client device, a command identifying a particular task to be performed. The device may process the command and the user activity data, with the trained machine learning models, to determine whether a particular action in the user activity data correlates with the particular task. The device may perform actions when the particular action correlates with the particular task.

## Claims

### Claim 1

A method, comprising:
receiving, by a device and from a client device associated with a user, user personalized data about the user and user activity data identifying tasks of the user and actions performed by the user in response to the tasks;
performing, by the device, natural language processing on the user personalized data and the user activity data to generate processed textual data identifying the user personalized data, the tasks, and the actions;
training, by the device, one or more machine learning models based on the processed textual data to generate one or more trained machine learning models that identify particular actions based on particular tasks;
receiving, by the device and from the client device, a command identifying a particular task to be performed;
processing, by the device, the command and the user activity data, with the one or more trained machine learning models, to determine whether a particular action in the user activity data correlates with the particular task; and
performing, by the device, one or more actions based on the particular action and when the particular action in the user activity data correlates with the particular task.

### Claim 2 (depends on 1)

The method of claim 1, wherein the command is a textual command and processing the command and the user activity data with the one or more trained machine learning models comprises:
processing the textual command, with a first trained machine learning model of the one or more trained machine learning models, to identify an intended task of the textual command; and
processing the intended task and the user activity data, with a second trained machine learning model of the one or more trained machine learning models, to determine whether a particular action in the user activity data correlates with the intended task.

### Claim 3 (depends on 1)

The method of claim 1, wherein performing the one or more actions comprises one of:
causing the particular action to be automatically implemented; or
processing the particular action, with a natural language generation model, to convert the particular action into a natural language response and providing the natural language response to the client device.

### Claim 4 (depends on 1)

The method of claim 1, wherein performing the one or more actions comprises one or more of:
providing, to the client device, instructions for performing the particular action;
automatically causing the client device to perform the particular action; or
receiving and analyzing feedback associated with performance of the particular action.

### Claim 5 (depends on 1)

The method of claim 1, wherein performing the one or more actions comprises:
automatically dispatching a robot, an unmanned vehicle, or a technician to perform the particular action;
updating a data structure based on feedback associated with performance of the particular action,
wherein the data structure stores the processed textual data; or
retraining the one or more machine learning models based on the particular action and based on feedback associated with performance of the particular action.

### Claim 6 (depends on 1)

The method of claim 1, wherein the command is a text-based command and processing the command comprises:
performing natural language processing on the text-based command to identify an intent of text-based command.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
receiving an image of a user-provided object prior to receipt of the command or with receipt of the command;
performing image recognition of the image of the user-provided object; and
determining whether the user is authenticated based on performing the image recognition of the image of the user-provided object.

### Claim 8

A device, comprising:
one or more memories; and
one or more processors, communicatively coupled to the one or more memories, configured to:
receive, from a client device associated with a user, a command identifying a particular task to be performed;
process the command and user activity data identifying tasks of the user and actions performed by the user in response to the tasks, with one or more machine learning models, to determine whether a particular action in the user activity data correlates with the particular task,
wherein the one or more machine learning models are trained to identify particular actions based on:
particular tasks,
user personalized data about the user, and
the user activity data; and
perform one or more actions based on the particular action and when the particular action in the user activity data correlates with the particular task.

### Claim 9 (depends on 8)

The device of claim 8, wherein the one or more processors are further configured to:
determine that the particular action in the user activity data fails to correlate with the particular task; and
provide, to the client device, a request to provide a new action for satisfying the particular task.

### Claim 10 (depends on 9)

The device of claim 9, wherein the one or more processors are further configured to:
receive, from the client device, information indicating the new action for satisfying the particular task; and
cause the new action to be automatically implemented.

### Claim 11 (depends on 9)

The device of claim 9, wherein the one or more processors are further configured to:
receive, from the client device, information indicating the new action for satisfying the particular task;
update a data structure based on the new action and based on performance of the new action,
wherein the data structure stores the user activity data; and
retrain the one or more machine learning models based on the new action and based on feedback associated with performance of the new action.

### Claim 12 (depends on 8)

The device of claim 8, wherein the user personalized data includes data identifying one or more of:
inbound email data associated with the user,
outbound email data associated with the user,
calendar data associated with the user,
browser history data associated with the user,
system data associated with the user,
software data associated with the user,
voice data associated with a voice of the user,
image data associated with a face of the user,
data identifying reasons for which the user communicates with other users,
data identifying frequent contacts of the user, or
data identifying an organizational hierarchy associated with the user.

### Claim 13 (depends on 8)

The device of claim 8, wherein the one or more processors are further configured to:
store the user personalized data and the user activity data in a data structure that is updated based on user actions and feedback associated with performance of the user actions.

### Claim 14 (depends on 8)

The device of claim 8, wherein the one or more machine learning models include one or more of:
a clustering model,
a decision tree model,
a k-means model,
a natural language processing model, or
a natural language generation model.

### Claim 15

A non-transitory computer-readable medium storing instructions, the instructions comprising:
one or more instructions that, when executed by one or more processors, cause the one or more processors to:
receive, from a client device associated with a user, user personalized data about the user and user activity data identifying tasks of the user and actions performed by the user in response to the tasks;
perform natural language processing on the user personalized data and the user activity data to generate processed textual data identifying the user personalized data, the tasks, and the actions;
train one or more machine learning models based on the processed textual data to generate one or more trained machine learning models that identify particular actions based on particular tasks;
receive, from the client device, a command identifying a particular task to be performed;
process the command, with a first trained machine learning model of the one or more trained machine learning models, to identify an intended task of the command;
process the intended task and the user activity data, with a second trained machine learning model of the one or more trained machine learning models, to determine whether a particular action in the user activity data correlates with the intended task; and
perform one or more actions based on the particular action and when the particular action in the user activity data correlates with the particular task.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, that cause the one or more processors to perform the one or more actions, cause the one or more processors to one of:
cause the particular action to be automatically implemented; or
process the particular action, with a natural language generation model, to convert the particular action into a natural language response and providing the natural language response to the client device.

### Claim 17 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, that cause the one or more processors to perform the one or more actions, cause the one or more processors to one or more of:
provide, to the client device, instructions for performing the particular action;
automatically cause the client device to perform the particular action;
receive and analyzing feedback associated with performance of the particular action;
automatically dispatch a robot, an unmanned vehicle, or a technician to perform the particular action;
update a data structure based on feedback associated with performance of the particular action,
wherein the data structure stores the processed textual data; or
retrain the one or more machine learning models based on the particular action and based on feedback associated with performance of the particular action.

### Claim 18 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:
receive an image of a face of the user prior to receipt of the command or with receipt of the command;
perform facial recognition of the image of the face or voice recognition of the command; and
determine whether the user is authenticated based on performing the facial recognition of the image of the face or the voice recognition of the command.

### Claim 19 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:
determine that the particular action in the user activity data fails to correlate with the particular task;
provide, to the client device, a request to provide a new action for satisfying the particular task;
receive, from the client device, information indicating the new action for satisfying the particular task; and
cause the new action to be automatically implemented.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:
store the user personalized data and the user activity data in a data structure that is updated based on user actions and feedback associated with performance of the user actions.
