US 12,456,465 B2Granted

Systems and methods for processing and presenting conversations

AssigneeOtter.ai, Inc.

InventorsYun Fu, Simon Lau, Fuchun Peng, Kaisuke Nakajima, Julius Cheng, Gelei Chen, Sam Song Liang

Life of this patent

Estimating the term…

Loading patent…

Abstract

System and method for processing and presenting a conversation. For example, a system includes a sensor configured to capture an audio-form conversation, and a processor configured to automatically transform the audio-form conversation into a transformed conversation. The transformed conversation includes a synchronized text, and the synchronized text is synchronized with the audio-form conversation. Additionally, the system includes a presenter configured to present the transformed conversation including the synchronized text and the audio-form conversation.

Claims13

Dependent claims

A system for processing and presenting a conversation, the system comprising:

a processor configured to:
receive a live audio-form conversation involving one or more speakers;
automatically transcribe the received live audio-form conversation into a live synchronized text, the live synchronized text being synchronized with the live audio-form conversation;
and
after the live audio-form conversation has been transcribed into the live synchronized text,
automatically generate a plurality of segments of the live audio-form conversation, the plurality of segments of the live audio-form conversation including a first segment of the live audio-form conversation and a second segment of the live audio-form conversation;
and
automatically generate a plurality of segments of the live synchronized text, the plurality of segments of the live synchronized text including a first segment of the live synchronized text and a second segment of the live synchronized text;
and
a presenter configured to present the plurality of segments of the live synchronized text;
wherein the processor is further configured to:
after the live audio-form conversation has been transcribed into the live synchronized text,
identify an occurrence of a natural pause during the live audio-form conversation in which there is an absence of speech from the one or more speakers;
and
in response to the identified occurrence of the natural pause,
automatically segment the live audio-form conversation into the first segment of the live audio-form conversation and the second segment of the live audio-form conversation;
and
automatically segment the live synchronized text into the first segment of the live synchronized text and the second segment of the live synchronized text;
wherein the first segment of the live audio-form conversation and the second segment of the live audio-form conversation are next to each other and are spoken by a same speaker;
wherein the processor is further configured to:
automatically assign a first speaker label to the first segment of the live synchronized text;
and
automatically assign a second speaker label to the second segment of the live synchronized text;
wherein the presenter is further configured to:
present the first speaker label together with the first segment of the live synchronized text;
and
present the second speaker label together with the second segment of the live synchronized text;
wherein:
the first segment of the live synchronized text and the second segment of the live synchronized text are next to each other;
the first speaker label represents the same speaker;
the second speaker label represents the same speaker;
and
the first speaker label and the second speaker label are the same.

30 more elements

A computer-implemented method for processing and presenting a conversation, the method comprising:

receiving a live audio-form conversation involving one or more speakers;
automatically transcribing the received live audio-form conversation into a live synchronized text, the live synchronized text being synchronized with the live audio-form conversation;
after the live audio-form conversation has been transcribed into the live synchronized text,
automatically generating a plurality of segments of the live audio-form conversation, the plurality of segments of the live audio-form conversation including a first segment of the live audio-form conversation and a second segment of the live audio-form conversation;
and
automatically generating a plurality of segments of the live synchronized text, the plurality of segments of the live synchronized text including a first segment of the live synchronized text and a second segment of the live synchronized text;
and
presenting the plurality of segments of the live synchronized text;
wherein the automatically generating a plurality of segments of the live audio-form conversation and the automatically generating a plurality of segments of the live synchronized text include:
after the live audio-form conversation has been transcribed into the live synchronized text,
identifying an occurrence of a natural pause during the live audio-form conversation in which there is an absence of speech from the one or more speakers;
and
in response to the identified occurrence of the natural pause,
automatically segmenting the live audio-form conversation into the first segment of the live audio-form conversation and the second segment of the live audio-form conversation;
and
automatically segmenting the live synchronized text into the first segment of the live synchronized text and the second segment of the live synchronized text;
wherein the first segment of the live audio-form conversation and the second segment of the live audio-form conversation are next to each other and are spoken by a same speaker;
wherein the automatically generating a plurality of segments of the live audio-form conversation and automatically generating a plurality of segments of the live synchronized text further include:
automatically assigning a first speaker label to the first segment of the live synchronized text;
and
automatically assigning a second speaker label to the second segment of the live synchronized text;
wherein the presenting the plurality of segments of the live synchronized text includes:
presenting the first speaker label together with the first segment of the live synchronized text;
and
presenting the second speaker label together with the second segment of the live synchronized text;
wherein:
the first segment of the live synchronized text and the second segment of the live synchronized text are next to each other;
the first speaker label represents the same speaker;
the second speaker label represents the same speaker;
and
the first speaker label and the second speaker label are the same.

28 more elements

A non-transitory computer-readable medium with instructions stored thereon, that when executed by a processor, perform the processes comprising:

receiving a live audio-form conversation involving one or more speakers;
automatically transcribing the received live audio-form conversation into a live synchronized text, the live synchronized text being synchronized with the live audio-form conversation;
after the live audio-form conversation has been transcribed into the live synchronized text,
automatically generating a plurality of segments of the live audio-form conversation, the plurality of segments of the live audio-form conversation including a first segment of the live audio-form conversation and a second segment of the live audio- form conversation;
and
automatically generating a plurality of segments of the live synchronized text, the plurality of segments of the live synchronized text including a first segment of the live synchronized text and a second segment of the live synchronized text;
and
presenting the plurality of segments of the live synchronized text;
wherein the automatically generating a plurality of segments of the live audio-form conversation and the automatically generating a plurality of segments of the live synchronized text include:
after the live audio-form conversation has been transcribed into the live synchronized text,
identifying an occurrence of a natural pause during the live audio-form conversation in which there is an absence of speech from the one or more speakers;
and
in response to the identified occurrence of the natural pause,
automatically segmenting the live audio-form conversation into the first segment of the live audio-form conversation and the second segment of the live audio-form conversation;
and
automatically segmenting the live synchronized text into the first segment of the live synchronized text and the second segment of the live synchronized text;
wherein the first segment of the live audio-form conversation and the second segment of the live audio-form conversation are next to each other and are spoken by a same speaker;
wherein the automatically generating a plurality of segments of the live audio-form conversation and the automatically generating a plurality of segments of the live synchronized text further include:
automatically assigning a first speaker label to the first segment of the live synchronized text;
and
automatically assigning a second speaker label to the second segment of the live synchronized text;
wherein the presenting the plurality of segments of the live synchronized text includes:
presenting the first speaker label together with the first segment of the live synchronized text;
and
presenting the second speaker label together with the second segment of the live synchronized text;
wherein:
the first segment of the live synchronized text and the second segment of the live synchronized text are next to each other;
the first speaker label represents the same speaker;
the second speaker label represents the same speaker;
and
the first speaker label and the second speaker label are the same.

28 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/131,982Filed 7 Apr 2023 · continuation
Priority: 9 Jul 2017
Published: US 2023/0245660 A13 Aug 2023
Granted: 28 Oct 2025
Term ends: Not estimatedThe source records don’t support an estimate.

Classification19 codes

G10L 15/26
Speech recognition: speech to text systems
H04N 7/155
Conference systems: involving storage of or access to video conference sessions
G06F 16/34
Of unstructured textual data: browsing; Visualisation therefor
G06F 16/35
Of unstructured textual data: clustering; Classification

Family

Application62/530,227Us provisional application · filed 9 Jul 2017
Application16/027,511Parent of this continuation · filed 5 Jul 2018
Application17/195,202Parent of this continuation · filed 8 Mar 2021
US 10,978,073 B1Parent of this continuation · filed 13 Apr 2021
US 2023/0245660 A1Pre-grant publication · published 3 Aug 2023
US 11,657,822 B2Related patent
US 2021/0217420 A1Published application
US 12,456,465 B2This patent

From the US filing history. Foreign family members aren’t listed.