System for automatic extraction of structure from spoken conversation using lexical and acoustic features

A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for extracting conversational structure from a spoken conversation, the method comprising:

• obtaining a voice record of the spoken conversation;
• dividing the voice record into at least three sequential utterances spoken by two different speakers;
• extracting a lexical feature from a respective utterance of the at least three sequential utterances in the voice record using speech recognition;
• extracting an acoustic feature from a respective utterance of the at least three sequential utterances in the voice record;
• and
• determining, via a machine learning method and based on the extracted lexical and acoustic features, a coarse-level conversational structure of the spoken conversation, which involves:
• determining a plurality of fine-level conversational activities based on the three sequential utterances, wherein a respective fine-level conversational activity is an activity identified based on respective words spoken by the two different speakers;
• determining, based on the plurality of fine-level conversational activities, a plurality of coarse-level conversational activities, wherein a respective coarse-level conversational activity of the plurality of coarse-level conversational activities indicates a phase of the spoken conversation that includes multiple fine-level conversational activities of the plurality of fine-level conversational activities;
• and
• generating the coarse-level conversational structure from the plurality of coarse-level conversational activities, wherein the coarse-level conversational structure indicates a high-level structure of the spoken conversation spanning the at least three sequential utterances.

A computing system for extracting conversational structure from a spoken conversation, the system comprising:

• a set of processors;
• and
• a non-transitory computer-readable medium coupled to the set of processors storing instructions thereon that, when executed by the processors, cause the processors to perform a method for extracting conversational structure from a spoken conversation, the method comprising:
• obtaining a voice record of the spoken conversation;
• dividing the voice record into at least three sequential utterances spoken by two different speakers;
• extracting a lexical feature from a respective utterance of the three sequential utterances in the voice record using an automatic speech recognition;
• extracting an acoustic feature from a respective utterance of the three sequential utterances the voice record;
• and
• determining, via a machine learning method and based on the extracted lexical feature and acoustic feature, a coarse-level conversational structure of the spoken conversation, which involves:
• determining a plurality of fine-level conversational activities based on the three sequential utterances, wherein a respective fine-level conversational activity is an activity identified based on respective words spoken by the two different speakers;
• determining, based on the plurality of fine-level conversational activities, a plurality of coarse-level conversational activities, wherein a respective coarse-level conversational activity of the plurality of coarse-level conversational activities indicates a phase of the spoken conversation that includes multiple fine-level conversational activities of the plurality of fine-level conversational activities;
• and
• generating the coarse-level conversational structure from the plurality of coarse-level conversational activities, wherein the coarse-level conversational structure indicates a high-level structure of the spoken conversation spanning the three sequential utterances.