US 11,379,671 B2Granted

Identification of response list

AssigneeSalesforce, Inc.

InventorsZachary Alexander, Edgar Gerardo Velasco, Victor Winslow Yee, Na Cheng, Khoa Le

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system is configured to analyze a corpus of historical chat data to identify the list of “best” responses. As such, the user is not required to identify a list of canned responses for input into the system. The described system uses a context word embedding function and response word embedding function to generate context vectors and response vectors corresponding to the corpus of conversation data, and the vectors are represented by a respective context matrix and a response matrix. The system processes these matrices to generate scores for responses, clusters the responses, and identifies the responses corresponding to the best scores for each cluster.

Claims21

Dependent claims

A method of processing a corpus of conversation data to automatically generate a set of candidate responses, comprising:

determining a set of similarity metrics for a response vector of a response matrix that comprises a plurality of response vectors by comparing the response vector to other response vectors in the response matrix, wherein the response vector is generated using a response comprising an input from the corpus of conversation data that is subsequent to a set of inputs in the corpus of conversation data comprising a context;
determining a set of fitness metrics for the response vector based at least in part on determining a probability that the response vector corresponds to a set of context vectors from a context matrix, wherein the set of context vectors is different from a context vector associated with the response vector and corresponding to the context in the corpus of conversation data;
determining a coherence score for the response vector based at least in part on the set of similarity metrics and the set of fitness metrics for the response vector;
determining a second set of similarity metrics, a second set of fitness metrics, and a second coherence score for one or more of the other response vectors from the response matrix;
clustering response vectors from the response matrix into a set of clusters based at least in part on similarities between the response vectors;
selecting a set of candidate response vectors from at least one cluster of the set of clusters based at least in part on coherence scores associated with response vectors in the at least one cluster;
and
identifying the set of candidate responses based at least in part on the selected set of candidate response vectors.

5 more elements

An apparatus for processing a corpus of conversation data to automatically generate a set of candidate responses, comprising:

a processor,
memory coupled with the processor;
and
instructions stored in the memory and executable by the processor to cause the apparatus to:
determine a set of similarity metrics for a response vector of a response matrix that comprises a plurality of response vectors by comparing the response vector to other response vectors in the response matrix, wherein the response vector is generated using a response comprising an input from the corpus of conversation data that is subsequent to a set of inputs in the corpus of conversation data comprising a context;
determine a set of fitness metrics for the response vector based at least in part on determining a probability that the response vector corresponds to a set of context vectors from a context matrix, wherein the set of context vectors is different from a context vector associated with the response vector and corresponding to the context in the corpus of conversation data;
determine a coherence score for the response vector based at least in part on the set of similarity metrics and the set of fitness metrics for the response vector;
determine a second set of similarity metrics, a second set of fitness metrics, and a second coherence score for one or more of the other response vectors from the response matrix;
cluster response vectors from the response matrix into a set of clusters based at least in part on similarities between the response vectors;
select a set of candidate response vectors from at least one cluster of the set of clusters based at least in part on coherence scores associated with response vectors in the at least one cluster;
and
identify the set of candidate responses based at least in part on the selected set of candidate response vectors.

9 more elements

A non-transitory computer-readable medium storing code for processing a corpus of conversation data to automatically generate a set of candidate responses, the code comprising instructions executable by a processor to:

determine a set of similarity metrics for a response vector of a response matrix that comprises a plurality of response vectors by comparing the response vector to other response vectors in the response matrix, wherein the response vector is generated using a response comprising an input from the corpus of conversation data that is subsequent to a set of inputs in the corpus of conversation data comprising a context;
determine a set of fitness metrics for the response vector based at least in part on determining a probability that the response vector corresponds to a set of context vectors from a context matrix, wherein the set of context vectors is different from a context vector associated with the response vector and corresponding to the context in the corpus of conversation data;
determine a coherence score for the response vector based at least in part on the set of similarity metrics and the set of fitness metrics for the response vector;
determine a second set of similarity metrics, a second set of fitness metrics, and a second coherence score for one or more of the other response vectors from the response matrix;
cluster response vectors from the response matrix into a set of clusters based at least in part on similarities between the response vectors;
select a set of candidate response vectors from at least one cluster of the set of clusters based at least in part on coherence scores associated with response vectors in the at least one cluster;
and
identify the set of candidate responses based at least in part on the selected set of candidate response vectors.

5 more elements

Dependencies are inferred from the claim text.