# US 11,310,173 B2 · Virtual agent chat model updates

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,310,173 B2
- Document ID: US11310173B2
- Status: Granted
- Kind: B2
- Application number: 16/427,623
- Filed: 2019-05-31
- Published: 2020-12-03
- Granted: 2022-04-19
- Earliest claimed: 2019-05-31
- Assignees: Kyndryl, Inc.
- Applicants: Kyndryl, Inc.
- Inventors: Garfield W. Vaughn; Vasanthi M. Gopal; Gandhi Sivakumar; Lakisha R. S. Hall
- Web page: https://patentgrep.com/patent/US11310173B2

## CPC

- G06F 17/40: Data acquisition and logging (for input to computer G06F3/00)
- H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
- G06F 11/0706: {the processing taking place on a specific hardware platform or in a specific software environment}
- G06F 11/0751: {Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06)}
- G06F 11/0763: {by bit configuration check, e.g. of formats or tags}
- G06F 11/0793: {Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40)}
- G06F 11/3476: {Data logging (G06F11/14, G06F11/2205 take precedence)}
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 20/00: Machine learning
- H04L 12/1827: {Network arrangements for conference optimisation or adaptation}
- H04L 12/1831: {Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status}
- H04L 51/04: Real-time or near real-time messaging, e.g. instant messaging [IM]
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

According to a computer-implemented method, a chat data set is received, which chat data set includes information indicative of a plurality of natural language chat transcripts of chats that occurred between a virtual agent and a human. Machine logic analyzes the chat data set to identify an error that occurred in the operation of the virtual agent. The machine logic updates a chat model based on the chat data set.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving a chat data set including information indicative of a plurality of natural language chat transcripts of chats that occurred between a virtual agent and a human;
determining a topic for a chat;
identifying, by machine logic and based on a determined topic for the chat, to which node in the chat flow the chat should be directed;
analyzing, by machine logic, the chat data set to identify an error that occurred during operation of the virtual agent based on the virtual agent incorrectly routing the chat to a node associated with a topic different from a topic of user input;
associating the error with the node where the error occurred;
identifying, by keyword analysis, that there is no node associated with the topic being discussed; and
updating, by machine logic, a chat model:
based on the chat data set; and
by generating a new node having subject matter expertise for the topic of the chat.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising receiving a transcript of chats that occurred between a human agent and the human.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, further comprising, analyzing, by machine logic, the transcript of chats that occurred between the human agent and the human.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein updating, by machine logic, the chat model, is further based on the transcript of chats that occurred between the human agent and the human.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein updating, by machine logic, the chat model comprises creating a new chat model.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein updating, by machine logic, the chat model comprises updating an existing chat model.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein updating the existing chat model comprises updating rules of the existing chat model, and wherein the rules indicate which node of the virtual agent a conversation is handed off to.

### Claim 8 (depends on 6)

The computer-implemented method of claim 6, wherein updating the existing chat model comprises updating information presented to a user via the virtual agent.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein identifying an error that occurred in the operation of the virtual agent is based on user input.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, further comprising flagging portions of the chat transcript to identify user dissatisfaction with an operation of the virtual agent and why the operation of the virtual agent was unsatisfactory.

### Claim 11

A system, comprising:
a log capture device to receive a chat data set including information indicative of a plurality of natural language chat transcripts of chats that occurred between a virtual agent and a human, wherein:
a virtual agent chat flow comprises a hierarchically organized arrangement of nodes; and
different nodes have different subject matter expertise and different natural language processing parameters and relate to different topics brought up in the chat;
a log analysis device to analyze the chat data set to:
identify when a human agent is introduced into the chat;
identify, as an error that occurred during operation of the virtual agent, when the virtual agent prematurely hands the chat off to the human agent;
associate the error with a node where the error occurred; and
identify, by keyword analysis, that there is no node associated with a topic being discussed; and
a chat model update device to update a chat model:
based on the chat data set by providing new keywords to indicate when a chat is to be passed to a different node; and
by generating a new node having subject matter expertise for the topic of the chat.

### Claim 12 (depends on 11)

The system of claim 11, wherein the log analysis device identifies when the virtual agent hands a conversation to an incorrect node of the virtual agent.

### Claim 13 (depends on 11)

The system of claim 11, wherein the log analysis device identifies when the virtual agent did not understand a conversation entry.

### Claim 14 (depends on 11)

The system of claim 11, wherein:
the log analysis device identifies variations in human correspondence requesting same information; and
the chat model update device updates the chat model based on an identified variation.

### Claim 15 (depends on 11)

The system of claim 11, further comprising a topic identifier to identify a topic of the chat data set.

### Claim 16 (depends on 15)

The system of claim 15:
further comprising a human log analysis device to analyze a transcript of chats that occurred between a human agent and the human; and
wherein the chat model update device updates the chat model based upon the transcript of chats that occurred between the human agent and the human.

### Claim 17 (depends on 11)

The system of claim 11, wherein the chat model update device updates rules that identify to which node of the virtual agent a conversation is handed.

### Claim 18 (depends on 11)

The system of claim 11, wherein the chat model update device adds nodes to a chat flow.

### Claim 19

A computer program product for managing a virtual agent, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor, to cause the processor to:
receive, by the processor, a chat data set including information indicative of a plurality of natural language chat transcripts of chats that occurred between a virtual agent and a human;
identify, by the processor, which nodes of the virtual agent are triggered in a chat and when triggered nodes of the virtual agent are triggered, wherein different nodes of the virtual agent have different subject matter expertise and different natural language processing parameters and relate to different topics brought up in the chat;
identify when a human agent is introduced into the chat to respond to a request;
analyze a transcript of the human agent responding to the request;
determine that a response by the virtual agent is unrelated to a topic of the chat; analyze, by the processor, the chat data set to identify an error that occurred during operation of the virtual agent based on:
the virtual agent incorrectly routing the chat to a node associated with a topic different from a topic of user input; and
the transcript of the human agent responding to the request;
identify, by the processor, the node in the virtual agent from which the error originates;
identify, by the processor, a human resolution of the error that occurred during operation of the virtual agent;
identify, by keyword analysis, that there is no node associated with the topic being discussed; and
automatically update, by the processor, a chat model based on the human resolution of the error by:
updating rules which identify to which node of the virtual agent the chat is to be handed;
providing new keywords to indicate when a chat is to be passed to a different node; and
creating a new node in an existing chat model wherein the new node has subject matter expertise for the topic of the chat.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the chat data set is added to a training dataset for a natural language processor.
