# US 10,482,883 B2 · Context-sensitive dynamic update of voice to text model in a voice-enabled electronic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,883 B2 - Document ID: US10482883B2 - Status: Granted - Kind: B2 - Application number: 15/969,291 - Filed: 2018-05-02 - Published: 2018-08-30 - Granted: 2019-11-19 - Earliest claimed: 2015-05-27 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Yuli Gao; Sangsoo Sung; Prathab Murugesan - Web page: https://patentgrep.com/patent/US10482883B2 ## CPC - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/065: Adaptation - G10L 15/083: {Recognition networks (G10L15/142, G10L15/16 take precedence)} - G10L 15/1822: {Parsing for meaning understanding} - G10L 2015/223: {Execution procedure of a spoken command} - G10L 2015/228: {of application context} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A voice to text model used by a voice-enabled electronic device is dynamically and in a context-sensitive manner updated to facilitate recognition of entities that potentially may be spoken by a user in a voice input directed to the voice-enabled electronic device. The dynamic update to the voice to text model may be performed, for example, based upon processing of a first portion of a voice input, e.g., based upon detection of a particular type of voice action, and may be targeted to facilitate the recognition of entities that may occur in a later portion of the same voice input, e.g., entities that are particularly relevant to one or more parameters associated with a detected type of voice action. ## Claims ### Claim 1 A method, comprising: receiving a voice input with a voice-enabled electronic device, the voice input including an original request that includes first and second portions, the second portion including a first context sensitive entity among a plurality of context sensitive entities that are associated with a context sensitive parameter and that potentially may be spoken in the voice input; and in the voice-enabled electronic device, and responsive to receiving the first portion of the voice input: performing local processing of the first portion of the voice input; determining during the local processing that the first portion is associated with the context sensitive parameter; and in response to determining that the first portion is associated with the context sensitive parameter, and prior to performing local processing of the second portion of the voice input including the first context sensitive entity: dynamically updating a local voice to text model, used by the voice-enabled electronic device, to augment the local voice to text model based on the context sensitive entities associated with the context sensitive parameter, wherein dynamically updating the local voice to text model facilitates recognition of the first context sensitive entity in performing local processing of the second portion of the voice input. ### Claim 2 (depends on 1) The method of claim 1, wherein performing the local processing includes: converting a digital audio signal of the voice input to text using a streaming voice to text module of the voice-enabled electronic device, wherein the streaming voice to text module dynamically generates a plurality of text tokens from the digital audio signal. ### Claim 3 (depends on 1) The method of claim 1, wherein the local voice to text model comprises at least one decoding graph, and wherein dynamically updating the local voice to text model to augment the local voice to text model based on the context sensitive entities includes adding a decoding path to the at least one decoding graph corresponding to each of the plurality of context sensitive entities. ### Claim 4 (depends on 1) The method of claim 1, further comprising, in response to a determination that the first portion is associated with the context sensitive parameter, prefetching from an online service voice to text model update data associated with the plurality of context sensitive entities, wherein dynamically updating the local voice to text model includes communicating the prefetched voice to text model update data to dynamically update the local voice to text model. ### Claim 5 (depends on 1) The method of claim 1, wherein determining during the local processing that the first portion is associated with the context sensitive parameter includes determining that the first portion includes a request to play a media item, wherein the context sensitive parameter includes a media item identifier for use in identifying the media item, and wherein the plurality of context sensitive entities identify a plurality of media items playable by the voice-enabled electronic device. ### Claim 6 (depends on 1) The method of claim 1, wherein determining during the local processing that the first portion is associated with the context sensitive parameter includes determining that the first portion includes a request to communicate with a contact, wherein the context sensitive parameter includes a contact identifier for use in initiating a communication with the contact, and wherein the plurality of context sensitive entities identify a plurality of contacts accessible by the voice-enabled electronic device. ### Claim 7 (depends on 1) The method of claim 1, wherein the context sensitive parameter is a location-dependent parameter, and wherein the plurality of context sensitive entities identify a plurality of points of interest disposed in proximity to a predetermined location. ### Claim 8 (depends on 7) The method of claim 7, wherein the predetermined location comprises a current location of the voice-enabled electronic device, the method further comprising, in response to a determination that the first portion is associated with the context sensitive parameter, communicating the current location to an online service and prefetching from the online service voice to text model update data associated with the plurality of context sensitive entities. ### Claim 9 A voice-enabled electronic device including memory and one or more processors operable to execute instructions stored in the memory, comprising instructions to: receive a voice input, the voice input including an original request that includes first and second portions, the second portion including a first context sensitive entity among a plurality of context sensitive entities that are associated with a context sensitive parameter and that potentially may be spoken in the voice input; and responsive to receiving the first portion of the voice input: perform local processing of the first portion of the voice input; determine during the local processing that the first portion is associated with the context sensitive parameter; and in response to determining that the first portion is associated with the context sensitive parameter, and prior to performing local processing of the second portion of the voice input including the first context sensitive entity: dynamically update a local voice to text model, used by the voice-enabled electronic device, to augment the local voice to text model based on the context sensitive entities associated with the context sensitive parameter, wherein dynamically updating the local voice to text model facilitates recognition of the first context sensitive entity in performing local processing of the second portion of the voice input. ### Claim 10 (depends on 9) The voice-enabled electronic device of claim 9, wherein the instructions to perform the local processing include instructions to: convert a digital audio signal of the voice input to text using a streaming voice to text module of the voice-enabled electronic device, wherein the streaming voice to text module dynamically generates a plurality of text tokens from the digital audio signal. ### Claim 11 (depends on 9) The voice-enabled electronic device of claim 9, wherein the local voice to text model comprises at least one decoding graph, and wherein dynamically updating the local voice to text model to augment the local voice to text model based on the context sensitive entities includes adding a decoding path to the at least one decoding graph corresponding to each of the plurality of context sensitive entities. ### Claim 12 (depends on 9) The voice-enabled electronic device of claim 9, further comprising instructions that, in response to a determination that the first portion is associated with the context sensitive parameter, prefetch from an online service voice to text model update data associated with the plurality of context sensitive entities, wherein the instructions to dynamically update the local voice to text model includes instructions to communicate the prefetched voice to text model update data to dynamically update the local voice to text model. ### Claim 13 (depends on 9) The voice-enabled electronic device of claim 9, wherein the instructions to determine during the local processing that the first portion is associated with the context sensitive parameter include instructions to determine that the first portion includes a request to play a media item, wherein the context sensitive parameter includes a media item identifier for use in identifying the media item, and wherein the plurality of context sensitive entities identify a plurality of media items playable by the voice-enabled electronic device. ### Claim 14 (depends on 9) The voice-enabled electronic device of claim 9, wherein the context sensitive parameter is a location-dependent parameter, and wherein the plurality of context sensitive entities identify a plurality of points of interest disposed in proximity to a predetermined location. ### Claim 15 (depends on 14) The voice-enabled electronic device of claim 14, wherein the predetermined location comprises a current location of the voice-enabled electronic device, and wherein the instructions further comprise instructions to, in response to a determination that the first portion is associated with the context sensitive parameter, communicate the current location to an online service and prefetch from the online service voice to text model update data associated with the plurality of context sensitive entities. ### Claim 16 A non-transitory computer readable storage medium storing computer instructions executable by one or more processors to perform a method comprising: receiving a voice input with a voice-enabled electronic device, the voice input including an original request that includes first and second portions, the second portion including a first context sensitive entity among a plurality of context sensitive entities that are associated with a context sensitive parameter and that potentially may be spoken in the voice input; and in the voice-enabled electronic device, and responsive to receiving the first portion of the voice input: performing local processing of the first portion of the voice input; determining during the local processing that the first portion is associated with the context sensitive parameter; and in response to determining that the first portion is associated with the context sensitive parameter, and prior to performing local processing of the second portion of the voice input including the first context sensitive entity: dynamically updating a local voice to text model, used by the voice-enabled electronic device, to augment the local voice to text model based on the context sensitive entities associated with the context sensitive parameter, wherein dynamically updating the local voice to text model facilitates recognition of the first context sensitive entity in performing local processing of the second portion of the voice input.