US 10,210,244 B1Granted

Updating natural language interfaces by processing usage data

AssigneeAsapp, Inc.

InventorsSatchuthananthavale Rasiah Kuhan Branavan, Joseph Ellsworth Hackman, Frederick William Poe Heckel, Aaron Isaksen

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

A third-party company may assist companies in providing natural language interfaces for their customers. To implement a natural language interface for a company, a configuration may be received that includes information, such as a list intents, seed messages for the intents, and hierarchical information of the intents. An intent classifier may be trained using the configuration, and the natural language interface may be deployed for use with customers. Usage data of the natural language classifier may be collected and used to improve the natural language interface. Messages corresponding to an intent may be clustered into clusters of similar messages, and a prototype message may be obtained for each cluster to provide a human understandable description of the cluster. The information about the clusters may be used to improve the natural language interface, such as by creating a new intent with a cluster or moving a cluster to a different intent.

Claims22

Dependent claims

A system for implementing natural language interfaces to allow customers to interact with companies, the system comprising:

at least one server computer comprising at least one processor and at least one memory, the at least one server computer configured to:
receive a configuration for a natural language interface for a company, wherein the configuration indicates a list of intents and a plurality of seed messages for each intent of the list of intents;
train an intent classifier using the configuration, wherein the intent classifier processes a message to classify the message as corresponding to one or more intents of the list of intents;
deploy the natural language interface using the intent classifier, wherein the natural language interface is used by customers of the company;
collect usage data for the natural language interface, wherein the usage data comprises a plurality of interactions, and wherein each interaction comprises a message received from one of a plurality of customers, an intent selected by the intent classifier, and an indication of success of the interaction;
analyze the usage data by:
obtaining a first set of interactions from the usage data, wherein the first set of interactions corresponds to a first intent of the list of intents,
clustering messages of the first set of interactions into a plurality of clusters, and
obtaining a prototype message for each cluster of the plurality of clusters, wherein each prototype message comprises an example message representing the associated cluster;
provide a user interface to view information about the plurality of clusters, the information comprising the prototype message for each cluster of the plurality of clusters;
receive an input to (i) create a new intent using a first cluster of the plurality of clusters or (ii) move the first cluster of the plurality of clusters to a second intent of the list of intents;
update the configuration using the received input;
update the intent classifier by retraining the intent classifier using the updated configuration and the usage data;
and
deploy the natural language interface using the updated intent classifier.

12 more elements

A computer-implemented method for modifying a customer service message, the method comprising:

receiving a configuration for a natural language interface for a company, wherein the configuration indicates a list of intents and a plurality of seed messages for each intent of the list of intents;
training an intent classifier using the configuration, wherein the intent classifier processes a message to classify the message as corresponding to one or more intents of the list of intents;
deploying the natural language interface using the intent classifier, wherein the natural language interface is used by customers of the company;
collecting usage data for the natural language interface, wherein the usage data comprises a plurality of interactions, and wherein each interaction comprises a message received from one of a plurality of customers, an intent selected by the intent classifier, and an indication of success of the interaction;
analyzing the usage data by:
obtaining a first set of interactions from the usage data, wherein the first set of interactions corresponds to a first intent of the list of intents,
clustering messages of the first set of interactions into a plurality of clusters, and
obtaining a prototype message for each cluster of the plurality of clusters, wherein each prototype message comprises an example message representing the associated cluster;
providing a user interface to view information about the plurality of clusters, the information comprising the prototype message for each cluster of the plurality of clusters;
receiving an input to (i) create a new intent using a first cluster of the plurality of clusters or (ii) move the first cluster of the plurality of clusters to a second intent of the list of intents;
updating the configuration using the received input;
updating the intent classifier by retraining the intent classifier using the updated configuration and the usage data;
deploying the natural language interface using the updated intent classifier.

10 more elements

One or more non-transitory computer-readable media comprising computer executable instructions that, when executed, cause at least one processor to perform actions comprising:

receiving a configuration for a natural language interface for a company, wherein the configuration indicates a list of intents and a plurality of seed messages for each intent of the list of intents;
training an intent classifier using the configuration, wherein the intent classifier processes a message to classify the message as corresponding to one or more intents of the list of intents;
deploying the natural language interface using the intent classifier, wherein the natural language interface is used by customers of the company;
collecting usage data for the natural language interface, wherein the usage data comprises a plurality of interactions, and wherein each interaction comprises a message received from one of a plurality of customers, an intent selected by the intent classifier, and an indication of success of the interaction;
analyzing the usage data by:
obtaining a first set of interactions from the usage data, wherein the first set of interactions corresponds to a first intent of the list of intents,
clustering messages of the first set of interactions into a plurality of clusters, and
obtaining a prototype message for each cluster of the plurality of clusters, wherein each prototype message comprises an example message representing the associated cluster;
providing a user interface to view information about the plurality of clusters, the information comprising the prototype message for each cluster of the plurality of clusters;
receiving an input to (i) create a new intent using a first cluster of the plurality of clusters or (ii) move the first cluster of the plurality of clusters to a second intent of the list of intents;
updating the configuration using the received input;
updating the intent classifier by retraining the intent classifier using the updated configuration and the usage data;
deploying the natural language interface using the updated intent classifier.

10 more elements

Dependencies are inferred from the claim text.