US 12,592,246 B2Granted

System and method for extracting hidden cues in interactive communications

AssigneeCapital One Services, LLC

InventorsAysu Ezen Can, Jan Amtrup

Life of this patent

Estimating the term…

Abstract

Disclosed herein are system, method, and computer program product embodiments for machine learning systems to process interactive communications between at least two participants. Speech and text, within the interactive communications, are analyzed using machine learning classifiers to extract prosodic, semantic and key phrase cues located within the interactive communications to identify changes to emotion, sentiments and key phrases. A summary of the interactive communications between a first participant and a second participant is generated at least, in-part, based on the extracted prosodic, semantic and key phrase cues and the summary is highlighted based on any of the changes to emotion, the sentiments or the key phrases.

Claims20

Dependent claims

A system for generating an interactive communication summary in a natural language processing environment, the system comprising:

a first machine learning model trained by a machine learning system, wherein the first machine learning model is configured to evaluate prosodic cues within an interactive communication between a first participant and a second participant to infer one or more changes to emotion of speech of the first participant;
a second machine learning model trained by the machine learning system, wherein the second machine learning model is configured to convert the interactive communication into text;
a third machine learning model trained by the machine learning system, wherein the third machine learning model is configured to evaluate, based on the one or more changes to the emotion of the speech of the first participant and the text, semantic cues in the interactive communication to infer one or more sentiments of the speech of the first participant;
a first trend detection algorithm configured to accumulate, in real-time, a plurality of the one or more sentiments to identify a sentiment trend in the interactive communication, wherein the first trend detection algorithm comprises an aspect-based sentiment classification on a turn level for each new utterance, and wherein the aspect comprises a service or product;
a fourth machine learning model trained by the machine learning system, wherein the fourth machine learning model is configured to evaluate the text of the interactive communication to infer one or more key phrases of the speech of the first participant, wherein the fourth machine learning model is trained with key phrases inferred in an unsupervised mode to recognize new phrasing;
a second trend detection algorithm configured, based on the new phrasing, to identify a phrasing trend in the interactive communication, wherein the phrasing trend correlates to a new issue with the service or product by the first participant;
and
a summarizer configured to:
generate a summary of the interactive communication;
and
highlight, in the summary, any of the one or more changes to:
the emotion of the speech, the one or more sentiments, the sentiment trend, the phrasing trend, or the one or more key phrases.

9 more elements

A computer-implemented method for processing a call in a natural language environment, comprising:

extracting and evaluating, by a first machine learning model, prosodic cues in an interactive communication between a first participant and a second participant to infer one or more changes to emotion of a speech of the first participant;
converting, by a second machine learning model, the interactive communication to text;
extracting and evaluating, by a third machine learning model and based on the one or more changes and additionally based on the text, semantic cues in the interactive communication to infer one or more sentiments of the speech of the first participant;
accumulating in real-time, by a first trend detection algorithm, a plurality of the one or more sentiments to identify a sentiment trend in the interactive communication, wherein the first trend detection algorithm comprises an aspect-based sentiment classification on a turn level for each new utterance, and wherein the aspect comprises a service or product;
evaluating, by a fourth machine learning model, the text of the interactive communication to extract one or more key phrases of the speech of the first participant, wherein the fourth machine learning model is trained with key phrases inferred in an unsupervised mode to recognize new phrasing;
accumulating, by a second trend detection algorithm, the new phrasing to identify a phrasing trend in the interactive communication, wherein the phrasing trend correlates to a new issue with the service or product by the first participant;
generating a summary of the interactive communication;
and
highlighting in the summary any of the one or more changes to the emotion of the speech, the one or more sentiments, the sentiment trend, or the one or more key phrases.

6 more elements

A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform natural language operations comprising:

extracting and evaluating, by a first machine learning model, prosodic cues in an interactive communication between a first participant and a second participant to infer one or more changes to emotion of a speech of the first participant;
converting, by a second machine learning model, the interactive communication to text;
extracting and evaluating, by a third machine learning model and based on the one or more changes and additionally based on the text, semantic cues in the interactive communication to infer one or more sentiments of the speech of the first participant;
accumulating in real-time, by a first trend detection algorithm, a plurality of the one or more sentiments to identify a sentiment trend in the interactive communication, wherein the first trend detection algorithm comprises an aspect-based sentiment classification on a turn level for each new utterance, and wherein the aspect comprises a service or product;
evaluating, by a fourth machine learning model, the text of the interactive communication to extract one or more key phrases of the speech of the first participant, wherein the fourth machine learning model is trained with key phrases inferred in an unsupervised mode to recognize new phrasing;
accumulating, by a second trend detection algorithm, the new phrasing to identify a phrasing trend in the interactive communication, wherein the phrasing trend correlates to a new issue with the service or product by the first participant;
and
generating a summary of the interactive communication;
and
highlighting in the summary any of the one or more changes to:
the emotion of the speech, the one or more sentiments, the sentiment trend, the phrasing trend, or the one or more key phrases.

8 more elements

Dependencies are inferred from the claim text.

patentgrep

Loading patent…

Abstract

A system for generating an interactive communication summary in a natural language processing environment, the system comprising:

a first machine learning model trained by a machine learning system, wherein the first machine learning model is configured to evaluate prosodic cues within an interactive communication between a first participant and a second participant to infer one or more changes to emotion of speech of the first participant;
a second machine learning model trained by the machine learning system, wherein the second machine learning model is configured to convert the interactive communication into text;
a third machine learning model trained by the machine learning system, wherein the third machine learning model is configured to evaluate, based on the one or more changes to the emotion of the speech of the first participant and the text, semantic cues in the interactive communication to infer one or more sentiments of the speech of the first participant;
a first trend detection algorithm configured to accumulate, in real-time, a plurality of the one or more sentiments to identify a sentiment trend in the interactive communication, wherein the first trend detection algorithm comprises an aspect-based sentiment classification on a turn level for each new utterance, and wherein the aspect comprises a service or product;
a fourth machine learning model trained by the machine learning system, wherein the fourth machine learning model is configured to evaluate the text of the interactive communication to infer one or more key phrases of the speech of the first participant, wherein the fourth machine learning model is trained with key phrases inferred in an unsupervised mode to recognize new phrasing;
a second trend detection algorithm configured, based on the new phrasing, to identify a phrasing trend in the interactive communication, wherein the phrasing trend correlates to a new issue with the service or product by the first participant;
and
a summarizer configured to:
generate a summary of the interactive communication;
and
highlight, in the summary, any of the one or more changes to:
the emotion of the speech, the one or more sentiments, the sentiment trend, the phrasing trend, or the one or more key phrases.

9 more elements