# US 12,223,950 B2 · Detecting conversations with computing devices > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,950 B2 - Document ID: US12223950B2 - Status: Granted - Kind: B2 - Application number: 18/144,694 - Filed: 2023-05-08 - Published: 2023-08-31 - Granted: 2025-02-11 - Earliest claimed: 2019-02-27 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Marcin Nowak-Przygodzki; Nathan David Howard; Gabor Simko; Andrei Giurgiu; Behshad Behzadi - Web page: https://patentgrep.com/patent/US12223950B2 ## CPC - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G06F 16/90332: {Natural language query formulation or dialogue systems} - G06F 16/9035: Filtering based on additional data, e.g. user or group profiles - G06F 16/909: using geographical or spatial information, e.g. location (spatiotemporally dependent retrieval from the web G06F16/9537) - G10L 15/07: to the speaker - G10L 15/08: Speech classification or search - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 25/51: for comparison or discrimination - G10L 15/1822: {Parsing for meaning understanding} - G10L 2015/088: {Word spotting} - G10L 2015/223: {Execution procedure of a spoken command} - G10L 2015/227: {of the speaker; Human-factor methodology} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for detecting a continued conversation are disclosed. In one aspect, a method includes the actions of receiving first audio data of a first utterance. The actions further include obtaining a first transcription of the first utterance. The actions further include receiving second audio data of a second utterance. The actions further include obtaining a second transcription of the second utterance. The actions further include determining whether the second utterance includes a query directed to a query processing system based on analysis of the second transcription and the first transcription or a response to the first query. The actions further include configuring the data routing component to provide the second transcription of the second utterance to the query processing system as a second query or bypass routing the second transcription. ## Claims ### Claim 1 A method implemented using one or more processors, comprising: receiving first audio data of a first utterance spoken by a user; obtaining a first transcription of the first utterance; determining whether the first utterance includes a query directed to a query processing system based on semantic analysis of the first transcription, wherein the semantic analysis of the first transcription includes processing the first transcription using, as a user model, a machine learning model that is trained based on examples of historical engagement of the user with the query processing system, wherein the examples of historical engagement of the user with the query processing system include training examples created by the user in response to one or more prompts issued by the query processing system, wherein the training examples are labeled by the user as being directed to the query processing system or not; based on determining whether the first utterance includes a query directed to the query processing system, configuring a data routing component to: provide the first transcription of the first utterance to the query processing system as a first query; or bypass routing the first transcription so that the first transcription is not provided to the query processing system. ### Claim 2 (depends on 1) The method of claim 1, wherein the user model comprises a transformer machine learning model. ### Claim 3 (depends on 1) The method of claim 1, wherein the examples of historical engagement of the user with the query processing system include one or more contexts when the user issued one or more queries of a log of past queries. ### Claim 4 (depends on 3) The method of claim 3, wherein the one or more contexts include one or more locations of the user when the user issued one or more of the queries of the log of past queries. ### Claim 5 (depends on 3) The method of claim 3, wherein the one or more contexts include one or more times of day of the user when the user issued one or more of the queries of the log of past queries. ### Claim 6 (depends on 1) The method of claim 1, wherein the examples of historical engagement of the user with the query processing system include one or more images of the user that were captured during past instances of the user issuing queries to the query processing system. ### Claim 7 A method implemented using one or more processors, comprising: receiving first audio data of a first utterance spoken by a user; obtaining a first transcription of the first utterance; determining whether the first utterance includes a query directed to a query processing system based on semantic analysis of the first transcription, wherein the semantic analysis of the first transcription includes processing the first transcription using a transformer machine learning model that is trained based on examples of historical engagement by one or more individuals with the query processing system, wherein the examples of historical engagement with the query processing system include training examples created by the user in response to one or more prompts issued by the query processing system, wherein the training examples are labeled by the user as being directed to the query processing system or not; based on determining whether the first utterance includes a query directed to the query processing system, configuring a data routing component to: provide the first transcription of the first utterance to the query processing system as a first query; or bypass routing the first transcription so that the first transcription is not provided to the query processing system. ### Claim 8 (depends on 7) The method of claim 7, wherein the examples of historical engagement with the query processing system include one or more contexts when the user issued one or more queries of a log of past queries. ### Claim 9 (depends on 8) The method of claim 8, wherein the one or more contexts include one or more locations of the user when the user issued one or more of the queries of the log of past queries. ### Claim 10 (depends on 8) The method of claim 8, wherein the one or more contexts include one or more times of day of the user when the user issued one or more of the queries of the log of past queries. ### Claim 11 (depends on 7) The method of claim 7, wherein the examples of historical engagement with the query processing system include one or more images of the user that were captured during past instances of the user issuing queries to the query processing system. ### Claim 12 A system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to: receive first audio data of a first utterance spoken by a user; obtain a first transcription of the first utterance; determine whether the first utterance includes a query directed to a query processing system based on semantic analysis of the first transcription, wherein the semantic analysis of the first transcription includes the first transcription being processed using, as a user model, a machine learning model that is trained based on examples of historical engagement of the user with the query processing system, wherein the examples of historical engagement of the user with the query processing system include training examples created by the user in response to one or more prompts issued by the query processing system, wherein the training examples are labeled by the user as being directed to the query processing system or not; based on a determination of whether the first utterance includes a query directed to the query processing system, configure a data routing component to: provide the first transcription of the first utterance to the query processing system as a first query; or bypass routing the first transcription so that the first transcription is not provided to the query processing system. ### Claim 13 (depends on 12) The system of claim 12, wherein the user model comprises a transformer machine learning model. ### Claim 14 (depends on 12) The system of claim 12, wherein the examples of historical engagement of the user with the query processing system include one or more contexts when the user issued one or more queries of a log of past queries. ### Claim 15 (depends on 14) The system of claim 14, wherein the one or more contexts include one or more locations of the user when the user issued one or more of the queries of the log of past queries. ### Claim 16 (depends on 14) The system of claim 14, wherein the one or more contexts include one or more times of day of the user when the user issued one or more of the queries of the log of past queries. ### Claim 17 (depends on 12) The system of claim 12, wherein the examples of historical engagement of the user with the query processing system include one or more images of the user that were captured during past instances of the user issuing queries to the query processing system.