# US 10,482,884 B1 · Outcome-oriented dialogs on a speech recognition platform > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,884 B1 - Document ID: US10482884B1 - Status: Granted - Kind: B1 - Application number: 15/663,514 - Filed: 2017-07-28 - Granted: 2019-11-19 - Earliest claimed: 2013-12-17 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Jeff Bradley Beal; Kevin Robert Charter; Ajay Gopalakrishnan; Sumedha Arvind Kshirsagar; Nishant Kumar - Web page: https://patentgrep.com/patent/US10482884B1 ## CPC - G10L 17/00: Speaker identification or verification techniques - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 2015/223: {Execution procedure of a spoken command} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A speech recognition platform configured to receive an audio signal that includes speech from a user and perform automatic speech recognition (ASR) on the audio signal to identify ASR results. The platform may identify: (i) a domain of a voice command within the speech based on the ASR results and based on context information associated with the speech or the user, and (ii) an intent of the voice command. In response to identifying the intent, the platform may perform multiple actions corresponding to this intent. The platform may select a target action to perform, and may engage in a back-and-forth dialog to obtain information for completing the target action. The action may include streaming audio to the device, setting a reminder for the user, purchasing an item on behalf of the user, making a reservation for the user or launching an application for the user. ## Claims ### Claim 1 A system comprising: one or more processors; and one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving first audio data representing a request; determining, based at least in part on the first audio data, first intent data representing a first intent associated with the request, the first intent associated with one or more slots; determining, based at least in part on the first audio data, second intent data representing a second intent associated with the request; determining a first number of values associated with the first intent; determining a second number of values associated with the second intent; selecting the first intent based at least in part on the first number of values being more favorable than the second number of values; determining that a slot of the one or more slots is unfilled; generating, based at least in part on the slot being unfilled, second audio data representing a query for additional information; sending the second audio data to a device to output audio corresponding to the second audio data; receiving third audio data representing a response to the query; and associating a value with the slot based at least in part on the third audio data. ### Claim 2 (depends on 1) The system of claim 1, wherein generating the second audio data is based at least in part on contextual data associated with the request. ### Claim 3 (depends on 1) The system of claim 1, wherein the first intent is associated with a domain and generating the second audio data is based at least in part on an activity associated with the domain. ### Claim 4 (depends on 1) The system of claim 1, wherein generating the second audio data is based at least in part on an estimated latency associated with generating the second audio data. ### Claim 5 (depends on 1) The system of claim 1, wherein generating the second audio data is based at least in part on historical data indicating previously-generated audio data associated with the first intent. ### Claim 6 A method comprising: receiving first audio data representing a request; determining, based at least in part on the first audio data, first intent data representing a first intent associated with the request, the first intent associated with one or more slots; determining, based at least in part on the first audio data, second intent data representing a second intent associated with the request; determining a first number of values associated with the first intent; determining a second number of values associated with the second intent; selecting the first intent based at least in part on the first number of values being more favorable than the second number of values; determining that a slot of the one or more slots is unfilled; generating, based at least in part on the slot being unfilled, second audio data representing a query for additional information; sending the second audio data to a device to output audio corresponding to the second audio data; receiving third audio data representing a response to the query; and associating a value with the slot based at least in part on the third audio data. ### Claim 7 (depends on 6) The method of claim 6, wherein generating the second audio data is based at least in part on contextual data associated with the request. ### Claim 8 (depends on 7) The method of claim 7, wherein the contextual information includes previous audio data representing a previous request received from the device. ### Claim 9 (depends on 6) The method of claim 6, further comprising sending, to the device, fourth audio data representing an indication of an action to be performed by the device. ### Claim 10 (depends on 6) The method of claim 6, wherein generating the second audio data is based at least in part on an estimated latency associated with generating the second audio data. ### Claim 11 (depends on 6) The method of claim 6, wherein the first intent corresponds to a domain and generating the second audio data is based at least in part on an activity associated with the domain. ### Claim 12 (depends on 6) The method of claim 6, wherein the request comprises a request to at least one of set a reminder, play a media file, add an item to a list, provide a recommendation, purchase an item, or place a telephone call. ### Claim 13 (depends on 6) The method of claim 6, wherein generating the second audio data is based at least in part on a type of value associated with the slot. ### Claim 14 (depends on 6) The method of claim 6, wherein generating the second audio data is based at least in part on an indication that causing output of the audio corresponding to the second audio data is more likely to fill the slot with the value than taking one or more other actions. ### Claim 15 A system comprising: one or more processors; and one or more computer-readable media including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving first audio data representing a request; determining, based at least in part on the first audio data and a context associated with the request, first intent data representing a first intent associated with the request, the first intent associated with one or more slots; determining, based at least in part on the first audio data and the context associated with the request, second intent data representing a second intent associated with the request; determining a first number of values associated with the first intent; determining a second number of values associated with the second intent; selecting the first intent based at least in part on the first number of values being more favorable than the second number of values; determining that a slot of the one or more slots is unfilled; generating, based at least in part on the slot being unfilled, second audio data representing a query for additional information; sending the second audio data to a device to output audio corresponding to the second audio data; receiving third audio data representing a response to the query; and associating a value with the slot based at least in part on the third audio data. ### Claim 16 (depends on 15) The system of claim 15, wherein generating the second audio data is based at least in part on contextual data associated with the request. ### Claim 17 (depends on 15) The system of claim 15, wherein the context includes a location of the device. ### Claim 18 (depends on 15) The system of claim 15, wherein generating the second audio data is based at least in part on a confidence value indicating that the response to the query will identify the value. ### Claim 19 (depends on 18) The system of claim 18, wherein generating the second audio data is based at least in part on the confidence value being at least a threshold confidence value. ### Claim 20 (depends on 15) The system of claim 15, wherein the first intent is associated with a domain and generating the second audio data is based at least in part on an activity associated with the domain.