# US 12,183,334 B2 · Customizing actions based on contextual data and voice-based inputs > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,334 B2 - Document ID: US12183334B2 - Status: Granted - Kind: B2 - Application number: 17/170,598 - Filed: 2021-02-08 - Published: 2021-05-27 - Granted: 2024-12-31 - Earliest claimed: 2013-03-01 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Zoltan Stekkelpak; Gyula Simonyi - Web page: https://patentgrep.com/patent/US12183334B2 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/07: to the speaker - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 2015/223: {Execution procedure of a spoken command} - G10L 2015/228: {of application context} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Methods and systems are provided for customizing an action. In some implementations, voice input is received from a user and a context is determined from the voice input. Potential contextual data is identified based on the context and the voice input. A level of confidence is determined for an association of the potential contextual data and the context. An action is performed based on the voice input, the potential contextual data, and the level of confidence. The potential contextual data is used to customize the action. ## Claims ### Claim 1 A computer-implemented method when executed on data processing hardware of a user device causes the data processing hardware to perform operations comprising: receiving a voice input command from a user to perform a specific action; processing the voice input command to determine a confidence that a home address of the user is associated with the specific action; accessing a user profile associated with the user to determine whether the user has granted access to the home address of the user; and when the user has granted access to the home address of the user: accessing, from the user profile, the home address of the user; determining that the confidence that the home address of the user is associated with the specific action fails to satisfy a confidence threshold; based on determining that the confidence that the home address of the user is associated with the specific action fails to satisfy the confidence threshold, presenting a refining prompt prompting the user to provide a confirmation response that confirms the home address of the user is associated with the specific action; receiving the confirmation response from the user; supplementing the voice input command with the home address of the user; and performing the specific action based on the supplemented voice input command. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the specific action is associated with a plurality of parameters, each parameter specifying information necessary to perform the specific action. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein: one of the plurality of parameters comprises the home address of the user; and processing the voice input command comprises processing the voice input command to determine that the one of the plurality of parameters that comprises the home address of the user is missing from the voice input command. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein processing the voice input command to determine that the home address of the user is necessary to perform the specific action comprises: processing the voice input command and the specific action to determine whether there is sufficient information to perform the specific action; and determining a missing parameter from the voice input command when there is not sufficient information, the missing parameter comprising the home address of the user. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein the operations further comprise identifying potential contextual data based on the missing parameter, the voice input command, and the specific action, the identified potential contextual data comprising the home address of the user. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the user profile comprises user-specific information input by the user, the user-specific information input by the user to the user profile comprises the home address of the user. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the user profile comprises privacy controls for allowing the user to grant or deny access to the home address of the user. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein presenting the refining prompt comprises presenting the refining prompt as an audio output from a speaker of a user device. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein presenting the refining prompt comprises displaying the refining prompt on a display of the user device. ### Claim 10 A system comprising: data processing hardware; and memory hardware in communication with the data processing and storing instructions that when executed by the data processing hardware cause the data processing hardware to perform operations comprising: receiving a voice input command from a user to perform a specific action; processing the voice input command to determine a confidence that a home address of the user is associated with the specific action; accessing a user profile associated with the user to determine whether the user has granted access to the home address of the user; and when the user has granted access to the home address of the user: accessing, from the user profile, the home address of the user; determining that the confidence that the home address of the user is associated with the specific action fails to satisfy a confidence threshold; based on determining that the confidence that the home address of the user is associated with the specific action fails to satisfy the confidence threshold, presenting a refining prompt prompting the user to provide a confirmation response that confirms the home address of the user is associated with the specific action; receiving the confirmation response from the user; supplementing the voice input command with the location-home address of the user; and performing the specific action based on the supplemented voice input command. ### Claim 11 (depends on 10) The system of claim 10, wherein the specific action is associated with a plurality of parameters, each parameter specifying information necessary to perform the specific action. ### Claim 12 (depends on 11) The system of claim 11, wherein: one of the plurality of parameters comprises the home address of the user; and processing the voice input command comprises processing the voice input command to determine that the one of the plurality of parameters that comprises the home address of the user is missing from the voice input command. ### Claim 13 (depends on 10) The system of claim 10, wherein processing the voice input command to determine that the home address of the user is necessary to perform the specific action comprises: processing the voice input command and the specific action to determine whether there is sufficient information to perform the specific action; and determining a missing parameter from the voice input command when there is not sufficient information, the missing parameter comprising the home address of the user. ### Claim 14 (depends on 13) The system of claim 13, wherein the operations further comprise identifying potential contextual data based on the missing parameter, the voice input command, and the specific action, the identified potential contextual data comprising the home address of the user. ### Claim 15 (depends on 10) The system of claim 10, wherein the user profile comprises user-specific information input by the user, the user-specific information input by the user to the user profile comprises the home address of the user. ### Claim 16 (depends on 10) The system of claim 10, wherein the user profile comprises privacy controls for allowing the user to grant or deny access to the home address of the user. ### Claim 17 (depends on 10) The system of claim 10, wherein presenting the refining prompt comprises presenting the refining prompt as an audio output from a speaker of a user device. ### Claim 18 (depends on 10) The system of claim 10, wherein presenting the refining prompt comprises displaying the refining prompt on a display of a user device.