# US 10,503,834 B2 · Template generation for a conversational agent

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,834 B2
- Document ID: US10503834B2
- Status: Granted
- Kind: B2
- Application number: 15/816,282
- Filed: 2017-11-17
- Published: 2019-05-23
- Granted: 2019-12-10
- Earliest claimed: 2017-11-17
- Assignees: Digital Genius Limited
- Applicants: Digital Genius Limited
- Inventors: Yoram Bachrach; Andrej {hacek over (Z)}ukov Gregori{hacek over (c)}; Samuel John Coope; Jose Marcos Rodríguez Fernández; Pavel Minkovsky; Bohdan Maksak
- Web page: https://patentgrep.com/patent/US10503834B2

## CPC

- G06F 40/30: Semantic analysis
- G06F 40/186: Templates
- G06F 40/284: Lexical analysis, e.g. tokenisation or collocates
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G10L 25/30: using neural networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Certain examples are described that provide methods and systems for generating templates for use by a conversational agent. These examples enable a natural language interface to be provided. Certain examples cluster user and agent messages from a corpus of text data representing text dialogues. This clustering enables response templates to be generated in a way that takes into account a context in which responses are given. In certain examples, messages that are exchanged between a user and a conversational agent are embedded as numeric arrays based a neural sequence-to-sequence model. Clustering routines are used to group dialogue encodings into one or more response clusters, and these clusters may then be used to generate response templates. The response templates may be used by a conversational agent to prepare a response to a user message.

## Claims

### Claim 1

A computer-implemented method for generating response templates for a conversational agent, the method comprising:
obtaining historical data representing a plurality of text dialogues, each text dialogue comprising a sequence of messages exchanged between a user and the conversational agent, each message comprising text data;
training a neural sequence-to-sequence model using the historical data,
the neural sequence-to-sequence model comprising an encoder and a decoder, the encoder receiving a sequence of tokens as an input and encoding the sequence of tokens as a numeric array, the decoder receiving the numeric array and generating another sequence of tokens as an output,
wherein said training includes, for a given text dialogue of the plurality of text dialogues:
supplying text from a dialogue prefix as the input to the neural sequence-to-sequence model, the dialogue prefix comprising at least a subset of the sequence of messages in the given text dialogue, and
supplying text from a next message following the dialogue prefix as a ground truth for comparison with the output of the neural sequence-to-sequence model;
generating dialogue encodings for agent response messages within the plurality of text dialogues using at least dialogue prefix encodings output by the encoder of the trained neural sequence-to-sequence model;
applying a clustering routine to group the plurality of dialogue encodings into one or more response clusters;
selecting a representative dialogue encoding for each response cluster; and
storing message text from the agent response messages associated with each representative dialogue encoding as a response template,
wherein the response templates are supplied to the conversational agent for use in generating an agent response message during a text dialogue of the plurality of text dialogues, and
wherein generating dialogue encodings comprises, for each text dialogue: encoding a dialogue prefix from the text dialogue using the encoder of the trained neural sequence-to-sequence model to generate a response context encoding;
encoding an agent response message following the dialogue prefix in the text dialogue as a numeric array to generate a response content encoding; and
fusing the response context encoding and response content encoding to generate a single numeric array for use as the dialogue encoding for the text dialogue.

### Claim 2 (depends on 1)

The method of claim 1, wherein fusing the response context encoding and response content encoding comprises normalizing each of the response context encoding and response content encoding and multiplying the normalized encodings by respective normalization ratio weights.

### Claim 3 (depends on 1)

The method of claim 1, wherein encoding an agent response message comprises:
tokenizing the text data of the agent response message into a sequence of tokens; and
computing a function of the sequence of tokens.

### Claim 4 (depends on 3)

The method of claim 3, wherein the function comprises at least one of:
a token count for tokens within a predefined fixed-length vocabulary array;
a term-frequency document-inverse-frequency count for tokens within a predefined fixed-length vocabulary array; and
a function of word embedding values for each of the sequence of tokens.

### Claim 5 (depends on 1)

The method of claim 1, wherein selecting a representative dialogue encoding for each cluster comprises:
determining a centroid of each cluster in the dialogue encoding space; and
for each centroid, selecting a dialogue encoding that minimizes a distance metric to the centroid.

### Claim 6 (depends on 1)

The method of claim 1, comprising:
receiving a set of messages for a text dialogue of the plurality of text dialogues;
selecting a response template from the response templates to generate an agent response message to reply to the last user message in the text dialogue; and
sending the agent response message to the user.

### Claim 7 (depends on 1)

The method of claim 1, wherein the encoder and the decoder of the sequence-to-sequence model comprise a recurrent neural network.

### Claim 8 (depends on 1)

The method of claim 1, wherein each message comprises text data in the form of a sequence of encoded characters that are stored in memory as a byte sequence.

### Claim 9

A system for implementing a text dialogue, comprising:
a conversational agent comprising at least a processor and a memory to receive one or more user messages from a client device over a network and send agent messages in response to the one or more user messages;
a template database comprising response templates for use by the conversational agent to generate the agent messages;
a dialogue database comprising historical data representing a plurality of text dialogues, each text dialogue comprising a sequence of messages exchanged between a user and a conversational agent, each message comprising text data;
a trained sequence-to-sequence model comprising an encoder and a decoder, the encoder receiving a sequence of tokens as an input and encoding the sequence of tokens as a numeric array, the decoder receiving the numeric array and generating another sequence of tokens as an output, the sequence-to-sequence model being trained using text from a dialogue prefix as the input, the dialogue prefix comprising at least a subset of the sequence of messages in a given text dialogue of the plurality of text dialogues, and text from a next message following the dialogue prefix as a ground truth for comparison with the output;
a dialogue encoding generator comprising at least a processor and a memory to generate dialogue encodings for agent response messages within the plurality of text dialogues using at least dialogue prefix encodings output by the encoder of the trained neural sequence-to-sequence model;
a clustering engine comprising at least a processor and a memory to group the plurality of dialogue encodings into one or more response clusters and output a representative dialogue encoding for each response cluster; and
a response template generator comprising at least a processor and a memory to receive representative dialogue encodings for the one or more response clusters and to store message text from the agent messages associated with each representative dialogue encoding as one of the response templates in the template database;
wherein the dialogue encoding generator further comprises:
a response context encoder to encode a dialogue prefix from a text dialogue of the plurality of text dialogues using the encoder of the trained neural sequence-to-sequence model to generate a response context encoding; and
a response content encoder to encode an agent response message following the dialogue prefix in the text dialogue as a numeric array to generate a response content encoding;
wherein the dialogue encoding generator is configured to fuse the response context encoding and response content encoding to generate a single numeric array for use as the dialogue encoding for the text dialogue.

### Claim 10 (depends on 9)

The system of claim 9, wherein the dialogue encoding generator is configured to weight and normalize the response context encoding and the response content encoding before fusing.

### Claim 11 (depends on 9)

The system of claim 9, wherein the clustering engine is configured to determine a centroid of each cluster in the dialogue encoding space and select a dialogue encoding for each centroid that minimizes a distance metric to the centroid.

### Claim 12 (depends on 9)

The system of claim 9, wherein the trained sequence-to-sequence model comprises numeric data representing parameter values for a recurrent neural network.

### Claim 13

A non-transient, computer-readable medium comprising computer program instructions that, when executed, cause a processor to:
obtain historical data representing a plurality of text dialogues, each text dialogue comprising a sequence of messages exchanged between a user and a conversational agent, each message comprising text data;
train a neural sequence-to-sequence model using the historical data,
the neural sequence-to-sequence model comprising an encoder and a decoder, the encoder receiving a sequence of tokens as an input and encoding the sequence of tokens as a numeric array, the decoder receiving the numeric array and generating another sequence of tokens as an output,
the instructions including instructions to, for a given text dialogue of the plurality of text dialogues:
use text from a dialogue prefix as the input to the neural sequence-to-sequence model, the dialogue prefix comprising at least a subset of the sequence of messages in the given text dialogue, and
use text from a next message following the dialogue prefix as a ground truth for comparison with the output of the neural sequence-to-sequence model;
generate dialogue encodings for agent response messages within the plurality of text dialogues using at least dialogue prefix encodings output by the encoder of the trained neural sequence-to-sequence model;
apply a clustering routine to group the plurality of dialogue encodings into one or more response clusters;
select a representative dialogue encoding for each response cluster; and
store message text from the agent response messages associated with each representative dialogue encoding as a response template,
wherein the response templates are supplied to the conversational agent for use in generating an agent response message during a text dialogue of the plurality of text dialogues, and
wherein generating dialogue encodings comprises, for each text dialogue:
encoding a dialogue prefix from the text dialogue using the encoder of the trained neural sequence-to-sequence model to generate a response context encoding;
encoding the agent response message following the dialogue prefix in the text dialogue as a numeric array to generate a response content encoding; and
fusing the response context encoding and response content encoding to generate a single numeric array for use as the dialogue encoding for the text dialogue.