# US 12,190,892 B2 · Selectively storing, with multiple user accounts and/or to a shared assistant device: speech recognition biasing, NLU biasing, and/or other data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,190,892 B2 - Document ID: US12190892B2 - Status: Granted - Kind: B2 - Application number: 18/381,417 - Filed: 2023-10-18 - Published: 2024-02-08 - Granted: 2025-01-07 - Earliest claimed: 2020-08-27 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Matthew Sharifi; Victor Carbune - Web page: https://patentgrep.com/patent/US12190892B2 ## CPC - G10L 17/22: Interactive procedures; Man-machine interfaces - G06V 40/172: {Classification, e.g. identification} - G10L 15/07: to the speaker - G10L 15/18: using natural language modelling - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 17/04: Training, enrolment or model building - G10L 17/06: Decision making techniques; Pattern matching strategies - G10L 17/00: Speaker identification or verification techniques - G10L 2015/223: {Execution procedure of a spoken command} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Some implementations relate to performing speech biasing, NLU biasing, and/or other biasing based on historical assistant interaction(s). It can be determined, for one or more given historical interactions of a given user, whether to affect future biasing for (1) the given user account, (2) additional user account(s), and/or (3) the shared assistant device as a whole. Some implementations disclosed herein additionally and/or alternatively relate to: determining, based on utterance(s) of a given user to a shared assistant device, an association of first data and second data; storing the association as accessible to a given user account of the given user; and determining whether to store the association as also accessible by additional user account(s) and/or the shared assistant device. ## Claims ### Claim 1 A method implemented by one or more processors of a shared assistant device, the method comprising: receiving, via one or more microphones of the shared assistant device, audio data that captures a spoken utterance of a user; generating, based on processing the audio data using a local speech-to-text (STT) engine of the shared assistant device, a transcription that corresponds to the spoken utterance captured in the audio data; resolving, based on processing the transcription using a local natural language understanding (NLU) engine of the shared assistant device, an assistant action to perform in response to receiving the spoken utterance; causing, in response to receiving the spoken utterance, performance of the assistant action resolved based on processing the transcription that corresponds to the spoken utterance; determining whether to store, locally at the shared assistant device, one or more NLU biasing parameters that are based on the assistant action resolved locally at the shared assistant device using the local NLU engine; in response to determining to store the one or more NLU biasing parameters locally at the shared assistant device: storing the one or more NLU biasing parameters locally at the shared assistant device; wherein storing the one or more NLU biasing parameters locally at the shared assistant device causes future spoken utterances, from any user and received at the shared assistant device, to be processed by the local NLU engine using the one or more NLU biasing parameters. ### Claim 2 (depends on 1) The method of claim 1, wherein resolving the assistant action to perform in response to receiving the spoken utterance comprises: determining, based on processing the transcription using the local NLU, a first candidate parameter and a conflicting second candidate parameter; prompting the user to disambiguate between the first candidate parameter and the second candidate parameter; and resolving, based on a response from the user to the prompting, the assistant action to include the first candidate parameter in lieu of the second candidate parameter. ### Claim 3 (depends on 2) The method of claim 2, wherein the NLU biasing parameters cause the local NLU engine to bias the future spoken utterances toward the first candidate parameter. ### Claim 4 (depends on 3) The method of claim 3, wherein causing performance of the assistant action comprises causing playing of a song and wherein the first candidate parameter is a song parameter for the song. ### Claim 5 (depends on 3) The method of claim 3, wherein causing performance of the assistant action comprises causing control of one or more smart devices and wherein the first candidate parameter is a parameter for the one or more smart devices. ### Claim 6 (depends on 3) The method of claim 3, wherein determining whether to store the one or more NLU biasing parameters in association with the shared assistant device comprises: classifying the spoken utterance as being provided by the user; classifying a previous spoken utterance as being provided by an additional user; and determining to store the one or more NLU biasing parameters in association with the shared assistant device based on the NLU biasing parameters being related to both the spoken utterance and the previous spoken utterance, and based on the spoken utterance and the previous spoken utterance being provided by both the user and the additional user. ### Claim 7 (depends on 1) The method of claim 1, wherein resolving the assistant action to perform in response to receiving the spoken utterance comprises: determining, based on processing the transcription using the local NLU, a first candidate intent and a conflicting second candidate intent; prompting the user to disambiguate between the first candidate intent and the second candidate intent; and resolving, based on a response from the user to the prompting, the assistant action to include the first candidate intent in lieu of the second candidate intent. ### Claim 8 (depends on 5) The method of claim 5, wherein the NLU biasing parameters cause the local NLU engine to bias the future spoken utterances toward the first candidate intent. ### Claim 9 (depends on 5) The method of claim 5, wherein determining whether to store the one or more NLU biasing parameters in association with the shared assistant device comprises: classifying the spoken utterance as being provided by the user; classifying a previous spoken utterance as being provided by an additional user; and determining to store the one or more NLU biasing parameters in association with the shared assistant device based on the NLU biasing parameters being related to both the spoken utterance and the previous spoken utterance, and based on the spoken utterance and the previous spoken utterance being provided by both the user and the additional user. ### Claim 10 (depends on 1) The method of claim 1, wherein determining whether to store the one or more NLU biasing parameters in association with the shared assistant device comprises: classifying the spoken utterance as being provided by the user; classifying a previous spoken utterance as being provided by an additional user; and determining to store the one or more NLU biasing parameters in association with the shared assistant device based on the NLU biasing parameters being related to both the spoken utterance and the previous spoken utterance, and based on the spoken utterance and the previous spoken utterance being provided by both the user and the additional user. ### Claim 11 A shared assistant device, comprising: microphones; memory storing instructions, a local speech-to-text (STT) engine, and a local natural language understanding (NLU) engine; and one or more processors operable to execute the instructions stored in the memory to: receive, via the microphones, audio data that captures a spoken utterance of a user; generate, based on processing the audio data using the local STT engine, a transcription that corresponds to the spoken utterance captured in the audio data; resolve, based on processing the transcription using the local NLU engine, an assistant action to perform in response to receiving the spoken utterance; cause, in response to receiving the spoken utterance, performance of the assistant action resolved based on processing the transcription that corresponds to the spoken utterance; determine whether to store, locally at the shared assistant device, one or more NLU biasing parameters that are based on the assistant action resolved locally at the shared assistant device using the local NLU engine; in response to determining to store the one or more NLU biasing parameters locally at the shared assistant device: store the one or more NLU biasing parameters locally at the shared assistant device; wherein storing the one or more NLU biasing parameters locally at the shared assistant device causes future spoken utterances, from any user and received at the shared assistant device, to be processed by the local NLU engine using the one or more NLU biasing parameters. ### Claim 12 (depends on 10) The shared assistant device of claim 10, wherein in resolving the assistant action to perform in response to receiving the spoken utterance one or more of the processors are to: determine, based on processing the transcription using the local NLU, a first candidate parameter and a conflicting second candidate parameter; prompt the user to disambiguate between the first candidate parameter and the second candidate parameter; and resolve, based on a response from the user to the prompting, the assistant action to include the first candidate parameter in lieu of the second candidate parameter. ### Claim 13 (depends on 11) The shared assistant device of claim 11, wherein the NLU biasing parameters cause the local NLU engine to bias the future spoken utterances toward the first candidate parameter. ### Claim 14 (depends on 13) The shared assistant device of claim 13, wherein in causing performance of the assistant action one or more of the processors are to cause playing of a song and wherein the first candidate parameter is a song parameter for the song. ### Claim 15 (depends on 13) The shared assistant device of claim 13, wherein in causing performance of the assistant action one or more of the processors are to cause control of one or more smart devices and wherein the first candidate parameter is a parameter for the one or more smart devices. ### Claim 16 (depends on 11) The shared assistant device of claim 11, wherein in resolving the assistant action to perform in response to receiving the spoken utterance one or more of the processors are to: determine, based on processing the transcription using the local NLU, a first candidate intent and a conflicting second candidate intent; prompt the user to disambiguate between the first candidate intent and the second candidate intent; and resolve, based on a response from the user to the prompting, the assistant action to include the first candidate intent in lieu of the second candidate intent. ### Claim 17 (depends on 16) The shared assistant device of claim 16, wherein the NLU biasing parameters cause the local NLU engine to bias the future spoken utterances toward the first candidate intent. ### Claim 18 (depends on 13) The shared assistant device of claim 13, wherein in causing performance of the assistant action one or more of the processors are to cause control of one or more smart devices. ### Claim 19 One or more non-transitory computer readable media storing instructions that, when executed by one or more processors, cause performance of a method comprising: receiving, via one or more microphones of the shared assistant device, audio data that captures a spoken utterance of a user; generating, based on processing the audio data using a local speech-to-text (STT) engine of the shared assistant device, a transcription that corresponds to the spoken utterance captured in the audio data; resolving, based on processing the transcription using a local natural language understanding (NLU) engine of the shared assistant device, an assistant action to perform in response to receiving the spoken utterance; causing, in response to receiving the spoken utterance, performance of the assistant action resolved based on processing the transcription that corresponds to the spoken utterance; determining whether to store, locally at the shared assistant device, one or more NLU biasing parameters that are based on the assistant action resolved locally at the shared assistant device using the local NLU engine; in response to determining to store the one or more NLU biasing parameters locally at the shared assistant device: storing the one or more NLU biasing parameters locally at the shared assistant device; wherein storing the one or more NLU biasing parameters locally at the shared assistant device causes future spoken utterances, from any user and received at the shared assistant device, to be processed by the local NLU engine using the one or more NLU biasing parameters.