# US 12,217,759 B2 · Voice to text conversion based on third-party agent content > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,759 B2 - Document ID: US12217759B2 - Status: Granted - Kind: B2 - Application number: 18/434,602 - Filed: 2024-02-06 - Published: 2024-08-15 - Granted: 2025-02-04 - Earliest claimed: 2016-12-07 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Barnaby James; Bo Wang; Sunil Vemuri; David Schairer; Ulas Kirazci; Ertan Dogrultan; Petar Aleksic - Web page: https://patentgrep.com/patent/US12217759B2 ## CPC - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G06F 40/205: Parsing - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06F 40/30: Semantic analysis - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 15/183: using context dependencies, e.g. language models - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - G10L 2015/223: {Execution procedure of a spoken command} - G10L 2015/228: {of application context} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Implementations relate to dynamically, and in a context-sensitive manner, biasing voice to text conversion. In some implementations, the biasing of voice to text conversions is performed by a voice to text engine of a local agent, and the biasing is based at least in part on content provided to the local agent by a third-party (3P) agent that is in network communication with the local agent. In some of those implementations, the content includes contextual parameters that are provided by the 3P agent in combination with responsive content generated by the 3P agent during a dialog that: is between the 3P agent, and a user of a voice-enabled electronic device; and is facilitated by the local agent. The contextual parameters indicate potential feature(s) of further voice input that is to be provided in response to the responsive content generated by the 3P agent. ## Claims ### Claim 1 A voice-enabled electronic device comprising: at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the at least one processor to be operable to: receive, from a third-party agent, one or more contextual parameters associated with the third-party agent, wherein the third-party agent is managed by an additional party that is distinct from the party that manages a local agent of the voice-enabled electronic device; receive, from a user of the voice-enabled electronic device, a voice input provided by the user; in response to receiving the voice input: convert, using a voice to text model, the voice input to text, wherein the instructions to convert the voice input to text comprise instructions to use one or more of the contextual parameters to bias the voice to text model in converting at least one segment of the voice input to the text; and transmit at least a portion of the text to the third-party agent. ### Claim 2 (depends on 1) The voice-enabled electronic device of claim 1, wherein one or more of the contextual parameters comprise one or more particular tokens. ### Claim 3 (depends on 1) The voice-enabled electronic device of claim 1, wherein one or more of the contextual parameters comprise one or more semantic types of tokens. ### Claim 4 (depends on 1) The voice-enabled electronic device of claim 1, wherein one or more of the semantic types of tokens identify one or more of a time semantic type and a date semantic type. ### Claim 5 (depends on 1) The voice-enabled electronic device of claim 1, wherein one or more of the contextual parameters comprise one or more semantic types of tokens and comprise one or more particular tokens. ### Claim 6 (depends on 1) The voice-enabled electronic device of claim 1, wherein the instructions further cause the voice-enabled electronic device to: receive, from the third-party agent and responsive to transmitting at least the portion of the text to the third-party agent, content that includes responsive content that is to be provided in response to the voice input; and provide the responsive content as output for presentation to the user via the voice-enabled electronic device, the output being provided in response to the voice input. ### Claim 7 (depends on 6) The voice-enabled electronic device of claim 6, wherein the instructions further cause the voice-enabled electronic device to: receive, from the user of the voice-enabled electronic device, an additional voice input provided by the user, the additional voice input being provided in response to the output; and use the content received from the third-party agent to convert the additional voice input to additional text. ### Claim 8 (depends on 7) The voice-enabled electronic device of claim 7, wherein the content received from the third-party agent comprises one or more additional contextual parameters associated with the third-party agent, and wherein using the content received from the third-party agent to convert the additional voice input to the additional text comprises using one or more of the additional contextual parameter to bias the voice to text model in converting at least one segment of the additional voice input to the additional text. ### Claim 9 (depends on 1) The voice-enabled electronic device of claim 1, wherein receiving one or more of the contextual parameters associated with the third-party agent is in response to an invocation of the third-party agent. ### Claim 10 (depends on 1) The voice-enabled electronic device of claim 1, wherein the voice to text model is a streaming voice to text model. ### Claim 11 A system comprising: at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the at least one processor to be operable to: receive, from a third-party agent, one or more contextual parameters associated with the third-party agent, wherein the third-party agent is managed by an additional party that is distinct from the party that manages a local agent of a voice-enabled electronic device; receive, from a user of the voice-enabled electronic device, a voice input provided by the user; in response to receiving the voice input: convert, using a voice to text model, the voice input to text, wherein the instructions to convert the voice input to text comprise instructions to use one or more of the contextual parameters to bias the voice to text model in converting at least one segment of the voice input to the text; and transmit at least a portion of the text to the third-party agent. ### Claim 12 (depends on 11) The system of claim 11, wherein one or more of the contextual parameters comprise one or more particular tokens. ### Claim 13 (depends on 11) The system of claim 11, wherein one or more of the contextual parameters comprise one or more semantic types of tokens. ### Claim 14 (depends on 11) The system of claim 11, wherein one or more of the semantic types of tokens identify one or more of a time semantic type and a date semantic type. ### Claim 15 (depends on 11) The system of claim 11, wherein one or more of the contextual parameters comprise one or more semantic types of tokens and comprise one or more particular tokens. ### Claim 16 (depends on 11) The system of claim 11, wherein the instructions further cause the voice-enabled electronic device to: receive, from the third-party agent and responsive to transmitting at least the portion of the text to the third-party agent, content that includes responsive content that is to be provided in response to the voice input; and provide the responsive content as output for presentation to the user via the voice-enabled electronic device, the output being provided in response to the voice input. ### Claim 17 (depends on 16) The system of claim 16, wherein the instructions further cause the voice-enabled electronic device to: receive, from the user of the voice-enabled electronic device, an additional voice input provided by the user, the additional voice input being provided in response to the output; and use the content received from the third-party agent to convert the additional voice input to additional text. ### Claim 18 (depends on 17) The system of claim 17, wherein the content received from the third-party agent comprises one or more additional contextual parameters associated with the third-party agent, and wherein using the content received from the third-party agent to convert the additional voice input to the additional text comprises using one or more of the additional contextual parameter to bias the voice to text model in converting at least one segment of the additional voice input to the additional text. ### Claim 19 (depends on 11) The system of claim 11, wherein receiving one or more of the contextual parameters associated with the third-party agent is in response to an invocation of the third-party agent. ### Claim 20 A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to be operable to perform operations, the operations comprising: receiving, from a third-party agent, one or more contextual parameters associated with the third-party agent, wherein the third-party agent is managed by an additional party that is distinct from the party that manages a local agent of a voice-enabled electronic device; receiving, from a user of the voice-enabled electronic device, a voice input provided by the user; in response to receiving the voice input: converting, using a voice to text model, the voice input to text, wherein the instructions to convert the voice input to text comprise instructions to use one or more of the contextual parameters to bias the voice to text model in converting at least one segment of the voice input to the text; and transmitting at least a portion of the text to the third-party agent.