US 12,500,856 B2Granted

Automated assistants with conference capabilities

AssigneeGoogle Inc.

InventorsMarcin Nowak-Przygodzki, Jan Lamecki, Behshad Behzadi

Life of this patent

Estimating the term…

Loading patent…

Abstract

Techniques are described related to enabling automated assistants to enter into a “conference mode” in which they can “participate” in meetings between multiple human participants and perform various functions described herein. In various implementations, an automated assistant implemented at least in part on conference computing device(s) may be set to a conference mode in which the automated assistant performs speech-to-text processing on multiple distinct spoken utterances, provided by multiple meeting participants, without requiring explicit invocation prior to each utterance. The automated assistant may perform semantic processing on first text generated from the speech-to-text processing of one or more of the spoken utterances, and generate, based on the semantic processing, data that is pertinent to the first text. The data may be output to the participants at conference computing device(s). The automated assistant may later determine that the meeting has concluded, and may be set to a non-conference mode.

Claims15

Dependent claims

A method implemented using one or more processors, comprising:

during a first conference dialog session between multiple participants, setting an automated assistant implemented at least in part on one or more conference computing devices to a conference mode in which the automated assistant performs semantic processing on multiple distinct spoken utterances without requiring explicit invocation of the automated assistant prior to each of the multiple distinct spoken utterances;
detecting, by the automated assistant based on the semantic processing of the multiple distinct utterances, one or more topics discussed by one or more of the multiple participants during the first conference dialog session;
storing one or more of the topics discussed by one or more of the multiple participants during the first conference dialog session for use during one or more subsequent conference dialog sessions, wherein the first conference dialog session and the one or more subsequent conference dialog sessions are distinct from one other;
subsequent to the storing, transitioning the automated assistant out of the conference mode;
determining that a second conference dialog session, in which the automated assistant is transitioned back into the conference mode, subsequent to the first conference dialog session is a follow up conference dialog session to the first conference dialog session, wherein the determining comprises analyzing one or more electronic calendar entries associated with the first and second conference dialog sessions to detect whether there is an overlap between the multiple participants of the first conference dialog session and at least one participant in the second conference dialog session that occurs subsequent to the first conference dialog session;
in response to detecting an overlap between the multiple participants of the first conference dialog session and at least one participant of the second conference dialog session, carrying over content of the first conference dialog session to the second conference dialog session, wherein the carrying over includes, and during the second conference dialog session that occurs subsequent to the first conference dialog session, retrieving one or more of the stored topics;
and
generating, by the automated assistant, data indicative of the one or more retrieved topics;
and
outputting, by the automated assistant at one or more of the conference computing devices, the data indicative of the one or more retrieved topics.

7 more elements

A non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by a processor, cause the processor to:

during a first conference dialog session between multiple participants, set an automated assistant implemented at least in part on one or more conference computing devices to a conference mode in which the automated assistant performs semantic processing on multiple distinct spoken utterances without requiring explicit invocation of the automated assistant prior to each of the multiple distinct spoken utterances;
detect, by the automated assistant based on the semantic processing of the multiple distinct utterances, one or more topics discussed by one or more of the multiple participants during the first conference dialog session;
store one or more of the topics discussed by one or more of the multiple participants during the first conference dialog session for use during one or more subsequent conference dialog sessions, wherein the first conference dialog session and the one or more subsequent conference dialog sessions are distinct from one other;
subsequent to the storing, transitioning the automated assistant out of the conference mode;
determine that a second conference dialog session, in which the automated assistant is transitioned back into the conference mode, subsequent to the first conference dialog session is a follow up conference dialog session to the first conference dialog session, wherein the instructions to determine comprise instructions to analyze one or more electronic calendar entries associated with the first and second conference dialog sessions to detect whether there is an overlap between the multiple participants of the first conference dialog session and at least one participant in the second conference dialog session that occurs subsequent to the first conference dialog session;
in response to detecting an overlap between the multiple participants of the first conference dialog session and at least one participant of the second conference dialog session, carry over content of the first conference dialog session to the second conference dialog session, wherein the instructions to carry over comprise instructions to, during the second conference dialog session that occurs subsequent to the first conference dialog session, retrieve one or more of the stored topics;
and
generate, by the automated assistant, data indicative of the one or more retrieved topics;
and
output, by the automated assistant at one or more of the conference computing devices, the data indicative of the one or more retrieved topics.

7 more elements

A system comprising one or more processors and memory storing instructions that, in response to execution of the instructions, cause the one or more processors to:

during a first conference dialog session between multiple participants, set an automated assistant implemented at least in part on one or more conference computing devices to a conference mode in which the automated assistant performs semantic processing on multiple distinct spoken utterances without requiring explicit invocation of the automated assistant prior to each of the multiple distinct spoken utterances;
detect, by the automated assistant based on the semantic processing of the multiple distinct utterances, one or more topics discussed by one or more of the multiple participants during the first conference dialog session;
store one or more of the topics discussed by one or more of the multiple participants during the first conference dialog session for use during one or more subsequent conference dialog sessions, wherein the first conference dialog session and the one or more subsequent conference dialog sessions are distinct from one other;
subsequent to the storing, transition the automated assistant out of the conference mode;
determine that a second conference dialog session, in which the automated assistant is transitioned back into the conference mode, subsequent to the first conference dialog session is a follow up conference dialog session to the first conference dialog session, wherein the instructions to determine comprise instructions to analyze one or more electronic calendar entries associated with the first and second conference dialog sessions to detect whether there is an overlap between the multiple participants of the first conference dialog session and at least one participant in the second conference dialog session that occurs subsequent to the first conference dialog session;
in response to detecting an overlap between the multiple participants of the first conference dialog session and at least one participant of the second conference dialog session, carry over content of the first conference dialog session to the second conference dialog session, wherein the instructions to carry over comprise instructions to, during the second conference dialog session that occurs subsequent to the first conference dialog session, retrieve one or more of the stored topics;
and
generate, by the automated assistant, data indicative of the one or more retrieved topics;
and
output, by the automated assistant at one or more of the conference computing devices, the data indicative of the one or more retrieved topics.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/944,712Filed 14 Sep 2022 · continuation
Priority: 2 Nov 2017
Published: US 2023/0013581 A119 Jan 2023
Granted: 16 Dec 2025
Term ends: Not estimatedThe source records don’t support an estimate.

Classification25 codes

H04L 51/02
User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail: using…
G06F 3/167
Sound input; Sound output: audio in a user interface, e.g. using voice commands for navigating, audio feedback
G06Q 10/1093
Time management, e.g. calendars, reminders, meetings or time accounting: calendar-based scheduling for persons or groups
G10L 15/1815
Using natural language modelling: semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning

Family

Application62/580,982Us provisional application · filed 2 Nov 2017
Application15/833,454Parent of this continuation · filed 6 Dec 2017
Application16/832,637Parent of this continuation · filed 27 Mar 2020
US 10,645,035 B2Parent of this continuation · filed 5 May 2020
US 2023/0013581 A1Pre-grant publication · published 19 Jan 2023
US 11,470,022 B2Parent of this continuation
US 2019/0132265 A1Published application
US 2020/0236069 A1Published application
US 12,500,856 B2This patent

From the US filing history. Foreign family members aren’t listed.