Template generation for a conversational agent

A system for implementing a text dialogue, comprising:

• a conversational agent comprising at least a processor and a memory to receive one or more user messages from a client device over a network and send agent messages in response to the one or more user messages;
• a template database comprising response templates for use by the conversational agent to generate the agent messages;
• a dialogue database comprising historical data representing a plurality of text dialogues, each text dialogue comprising a sequence of messages exchanged between a user and a conversational agent, each message comprising text data;
• a trained sequence-to-sequence model comprising an encoder and a decoder, the encoder receiving a sequence of tokens as an input and encoding the sequence of tokens as a numeric array, the decoder receiving the numeric array and generating another sequence of tokens as an output, the sequence-to-sequence model being trained using text from a dialogue prefix as the input, the dialogue prefix comprising at least a subset of the sequence of messages in a given text dialogue of the plurality of text dialogues, and text from a next message following the dialogue prefix as a ground truth for comparison with the output;
• a dialogue encoding generator comprising at least a processor and a memory to generate dialogue encodings for agent response messages within the plurality of text dialogues using at least dialogue prefix encodings output by the encoder of the trained neural sequence-to-sequence model;
• a clustering engine comprising at least a processor and a memory to group the plurality of dialogue encodings into one or more response clusters and output a representative dialogue encoding for each response cluster;
• and
• a response template generator comprising at least a processor and a memory to receive representative dialogue encodings for the one or more response clusters and to store message text from the agent messages associated with each representative dialogue encoding as one of the response templates in the template database;
• wherein the dialogue encoding generator further comprises:
• a response context encoder to encode a dialogue prefix from a text dialogue of the plurality of text dialogues using the encoder of the trained neural sequence-to-sequence model to generate a response context encoding;
• and
• a response content encoder to encode an agent response message following the dialogue prefix in the text dialogue as a numeric array to generate a response content encoding;
• wherein the dialogue encoding generator is configured to fuse the response context encoding and response content encoding to generate a single numeric array for use as the dialogue encoding for the text dialogue.

A non-transient, computer-readable medium comprising computer program instructions that, when executed, cause a processor to:

• obtain historical data representing a plurality of text dialogues, each text dialogue comprising a sequence of messages exchanged between a user and a conversational agent, each message comprising text data;
• train a neural sequence-to-sequence model using the historical data,
• the neural sequence-to-sequence model comprising an encoder and a decoder, the encoder receiving a sequence of tokens as an input and encoding the sequence of tokens as a numeric array, the decoder receiving the numeric array and generating another sequence of tokens as an output,
• the instructions including instructions to, for a given text dialogue of the plurality of text dialogues:
• use text from a dialogue prefix as the input to the neural sequence-to-sequence model, the dialogue prefix comprising at least a subset of the sequence of messages in the given text dialogue, and
• use text from a next message following the dialogue prefix as a ground truth for comparison with the output of the neural sequence-to-sequence model;
• generate dialogue encodings for agent response messages within the plurality of text dialogues using at least dialogue prefix encodings output by the encoder of the trained neural sequence-to-sequence model;
• apply a clustering routine to group the plurality of dialogue encodings into one or more response clusters;
• select a representative dialogue encoding for each response cluster;
• and
• store message text from the agent response messages associated with each representative dialogue encoding as a response template,
• wherein the response templates are supplied to the conversational agent for use in generating an agent response message during a text dialogue of the plurality of text dialogues, and
• wherein generating dialogue encodings comprises, for each text dialogue:
• encoding a dialogue prefix from the text dialogue using the encoder of the trained neural sequence-to-sequence model to generate a response context encoding;
• encoding the agent response message following the dialogue prefix in the text dialogue as a numeric array to generate a response content encoding;
• and
• fusing the response context encoding and response content encoding to generate a single numeric array for use as the dialogue encoding for the text dialogue.