# US 12,008,983 B1 · User feedback for speech interactions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,983 B1 - Document ID: US12008983B1 - Status: Granted - Kind: B1 - Application number: 17/731,968 - Filed: 2022-04-28 - Granted: 2024-06-11 - Earliest claimed: 2013-01-11 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Gilles Jean Roger Belin; Charles S. Rogers, III; Robert David Owen; Jeffrey Penrod Adams; Rajiv Ramachandran; Gregory Michael Hart - Web page: https://patentgrep.com/patent/US12008983B1 ## CPC - G10L 15/00: Speech recognition (G10L17/00 takes precedence) - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/01: Assessment or evaluation of speech recognition systems - G10L 15/07: to the speaker - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 2015/221: {Announcement of recognition results} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract An interactive system may be implemented in part by an audio device located within a user environment, which may accept speech commands from a user and may also interact with the user by means of generated speech. In order to improve performance of the interactive system, a user may use a separate device, such as a personal computer or mobile device, to access a graphical user interface that lists details of historical speech interactions. The graphical user interface may be configured to allow the user to provide feedback and/or corrections regarding the details of specific interactions. ## Claims ### Claim 1 A method implemented at least in part by a computing device, the method comprising: generating first audio data representing speech of a user captured by one or more microphones of the computing device; sending, using a wireless interface of the computing device, the first audio data representing the speech of the user to one or more remote computing devices; receiving, from the one or more remote computing devices and via the wireless interface, text data representing text that differs from the speech of the user; presenting the text on a display of the computing device; presenting, on the display, a selectable control that, when selected, initiates a request to output the first audio data representing the speech of the user; receiving a selection of the selectable control; causing, based at least in part on the selection, a speaker of the computing device to output the first audio data representing the speech of the user; and sending, using the wireless interface, a request to delete data associated with the speech of the user to the one or more remote computing devices. ### Claim 2 (depends on 1) The method as recited in claim 1, wherein the selectable control comprises a first selectable control, and further comprising presenting, on the display, a second selectable control associated with providing feedback regarding at least one of the text or an action performed in response to the speech of the user. ### Claim 3 (depends on 1) The method as recited in claim 1, comprising: generating, prior to generating the first audio data, second audio data based at least in part on sound captured by the one or more microphones, the sound corresponding to a predefined utterance; and analyzing the second audio data to identify the predefined utterance. ### Claim 4 (depends on 1) The method as recited in claim 1, further comprising receiving input data via an input field presented on the display, the input data corresponding to the request to delete the data associated with the speech of the user. ### Claim 5 (depends on 4) The method as recited in claim 4, further comprising deleting the data associated with the speech of the user based at least in part on the input data. ### Claim 6 (depends on 1) The method as recited in claim 1, further comprising providing, based at least in part on input data representing a selection of the speech of the user, additional data for presentation on the display, the additional data associated with the speech of the user. ### Claim 7 (depends on 1) The method as recited in claim 1, further comprising updating, based at least in part on input data representing feedback associated with the text, at least one of a model for recognizing speech or a model for determining user intent. ### Claim 8 (depends on 1) The method as recited in claim 1, further comprising presenting, on the display, a plurality of prior utterances, the plurality of prior utterances presented in chronological order. ### Claim 9 A first computing device comprising: a display; a speaker; a wireless interface; one or more processors; and one or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising: receiving, via the wireless interface and from one or more remote computing devices, text data produced using audio data generated based at least in part on first sound captured by one or more microphones of a second computing device, the first sound corresponding to speech of a user, wherein the text data represents text that differs from the speech of the user; presenting the text on a display of the first computing device; presenting, on the display, a selectable control that, when selected, initiates a request to output the audio data generated based at least in part on the first sound corresponding to the speech of the user; receiving a selection of the selectable control; causing, based at least in part on the selection, the speaker to output the audio data generated based at least in part on the first sound corresponding to the speech of the user; and sending, using the wireless interface, a request to delete data associated with the speech of the user to the one or more remote computing devices. ### Claim 10 (depends on 9) The first computing device as recited in claim 9, wherein the selectable control comprises a first selectable control, and the one or more computer-readable media further store computer-executable instructions that, when executed, cause the one or more processors to perform an act comprising presenting, on the display, a second selectable control associated with providing feedback regarding at least one of the text or an action performed in response to the speech of the user. ### Claim 11 (depends on 9) The first computing device as recited in claim 9, further comprising receiving input data via an input field presented on the display, the input data corresponding to the request to delete the data associated with the speech of the user. ### Claim 12 (depends on 11) The first computing device as recited in claim 11, wherein the one or more computer-readable media further store computer-executable instructions that, when executed, cause the one or more processors to perform an act comprising deleting the data associated with the speech of the user based at least in part on the input data. ### Claim 13 (depends on 9) The first computing device as recited in claim 9, wherein the one or more computer-readable media further store computer-executable instructions that, when executed, cause the one or more processors to perform an act comprising providing, based at least in part on input data representing a selection of the speech of the user, additional data for presentation on the display, the additional data associated with the speech of the user. ### Claim 14 (depends on 9) The first computing device as recited in claim 9, wherein the one or more computer-readable media further store computer-executable instructions that, when executed, cause the one or more processors to perform an act comprising updating, based at least in part on input data representing feedback associated with the text, at least one of a model for recognizing speech or a model for determining user intent. ### Claim 15 (depends on 9) The first computing device as recited in claim 9, wherein the one or more computer-readable media further store computer-executable instructions that, when executed, cause the one or more processors to perform an act comprising presenting, on the display, a plurality of prior utterances, the plurality of prior utterances presented in chronological order. ### Claim 16 A method implemented at least in part by a computing device, the method comprising: generating first audio data representing speech of a user captured by one or more microphones of the computing device; sending, using a wireless interface of the computing device, the first audio data representing the speech of the user to one or more remote computing devices; receiving, from the one or more remote computing devices and via the wireless interface, text data representing text that differs from the speech of the user; presenting the text on a display of the computing device; presenting, on the display, a selectable control that, when selected, initiates a request to output the first audio data representing the speech of the user; receiving a selection of the selectable control; causing, based at least in part on the selection, a speaker of the computing device to output the first audio data representing the speech of the user; receiving feedback regarding at least one of the text or an action performed in response to the speech of the user; and sending, using the wireless interface, the feedback regarding at least one of the text or the action performed in response to the speech of the user. ### Claim 17 (depends on 16) The method as recited in claim 16, further comprising sending, using the wireless interface, a request to delete data associated with the speech of the user to the one or more remote computing devices. ### Claim 18 (depends on 16) The method as recited in claim 16, further comprising presenting, on the display, a plurality of prior utterances, the plurality of prior utterances presented in chronological order. ### Claim 19 (depends on 16) The method as recited in claim 16, wherein the selectable control comprises a first selectable control, and further comprising presenting, on the display, a second selectable control associated with providing the feedback. ### Claim 20 (depends on 16) The method as recited in claim 16, further comprising providing, based at least in part on input data representing a selection of the speech of the user, additional data for presentation on the display, the additional data associated with the speech of the user.