US 2021/0264909 A1 · DETERMINING CONVERSATION ANALYSIS INDICATORS FOR A MULTIPARTY CONVERSATION

Abstract

Technology is provided for conversation analysis. The technology includes, receiving multiple utterance representations, where each utterance representation represents a portion of a conversation performed by at least two users, and each utterance representation is associated with video data, acoustic data, and text data. The technology further includes generating a first utterance output by applying video data, acoustic data, and text data of the first utterance representation to a respective video processing part of the machine learning system to generate video, text, and acoustic-based outputs. A second utterance output is further generated for a second user. Conversation analysis indicators are generated by applying, to a sequential machine learning system the combined speaker features and a previous state of the sequential machine learning system.

Claims20

Dependent claims

A method to generate a conversation analysis, the method comprising:

receiving multiple utterance representations,
wherein each utterance representation represents a portion of a conversation performed by at least two users, and
wherein each utterance representation is associated with video data, acoustic data, and text data;
and
generating a first utterance output by applying a first utterance representation, that is associated with a first user and that is of the multiple utterance representations, to a machine learning system, wherein generating the first utterance output includes:
applying video data of the first utterance representation to a first video processing part of the machine learning system to generate first video-based output;
applying acoustic data of the first utterance representation to a first acoustic processing part of the machine learning system to generate first acoustic-based output;
applying text data of the first utterance representation to a first textual processing part of the machine learning system to generate first text-based output;
and
generating the first utterance output by combining data that is based on the first video-based output, the first acoustic-based output, and the first text-based output.

7 more elements

A computing system for generating conversation analysis indicators for a conversation performed by at least two users, the computing system comprising:

one or more processors;
and
one or more memories storing computer-executable instructions that, when executed by the one or more processors, cause the computing system to perform operations comprising:
applying video data of a first utterance representation from the conversation to a first video processing part of the machine learning system to generate first video-based output;
applying acoustic data of the first utterance representation to a first acoustic processing part of the machine learning system to generate first acoustic-based output;
applying text data of the first utterance representation to a first textual processing part of the machine learning system to generate first text-based output;
and
generating the first utterance output by combining data that is based on the first video-based output, the first acoustic-based output, and the first text-based output.

5 more elements

A computer-readable storage medium storing instructions that, when executed by one or more processors of a computing system, cause the computing system to perform actions comprising:

receiving a first utterance representation from a conversation performed by at least two users;
generating a first utterance output by applying the first utterance representation to a machine learning system, wherein the generating the first utterance output includes:
applying video data of the first utterance representation to a first video processing part of the machine learning system to generate first video-based output;
applying acoustic data of the first utterance representation to a first acoustic processing part of the machine learning system to generate first acoustic-based output;
and
generating the first utterance output by combining data that is based on the first video-based output and the first acoustic-based output.

3 more elements

Dependencies are inferred from the claim text.

Determining conversation analysis indicators for a multiparty conversation

Life of this patent

Determining conversation analysis indicators for a multiparty conversation

Life of this patent

Abstract

Claims20