# US 11,443,739 B1 · Connected accessory for a voice-controlled device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,739 B1 - Document ID: US11443739B1 - Status: Granted - Kind: B1 - Application number: 16/668,866 - Filed: 2019-10-30 - Granted: 2022-09-13 - Earliest claimed: 2016-11-11 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Derick Deller; Apoorv Naik; Zoe Adams; Aslan Appleman; Link Cornelius; Pete Klein - Web page: https://patentgrep.com/patent/US11443739B1 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 40/295: Named entity recognition - G06T 13/205: {driven by audio data} - G10L 13/047: Architecture of speech synthesisers - G10L 13/10: Prosody rules derived from text; Stress or intonation - G10L 15/142: {Hidden Markov Models [HMMs]} - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - H04L 67/02: based on web technology, e.g. hypertext transfer protocol [HTTP] - H04L 67/025: for remote control or remote monitoring of applications - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - H04L 67/306: User profiles - G10H 2210/076: for extraction of timing, tempo; Beat detection - G10L 13/00: Speech synthesis; Text to speech systems - G10L 2015/088: {Word spotting} - G10L 2015/223: {Execution procedure of a spoken command} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE ## Abstract Coordinated operation of a voice-controlled device and an accessory device in an environment is described. A remote system processes audio data it receives from the voice-controlled device in the environment to identify a first intent associated with a first domain, a second intent associated with a second domain, and a named entity associated with the audio data. The remote system sends, to the voice-controlled device, first information for accessing main content associated with the named entity, and a first instruction corresponding to the first intent. The remote system also sends, to the accessory device, second information for accessing control information or supplemental content associated with the main content, and a second instruction corresponding to the second intent. The first and second instructions, when processed by the devices in the environment, cause coordinated operation of the voice-controlled device and the accessory device. ## Claims ### Claim 1 A system comprising: one or more processors; and non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: receiving input data representing a request to output content on a first device associated with a user profile; determining, based at least in part on the user profile, that a second device has been configured to output a first portion of the content while the first device outputs a second portion of the content; sending, to the second device, first data representing the first portion of the content causing the second device to output the first portion of the content; and sending, to the first device, second data representing the second portion of the content causing the second device to output the first portion of the content in a time-synchronous manner with the second portion of the content output by the first device. ### Claim 2 (depends on 1) The system of claim 1, wherein the second data causes a display of the first device to present a visual representation of the first portion of the content in a time-synchronous manner with output of audio corresponding to the second portion of the content by a speaker of the second device. ### Claim 3 (depends on 1) The system of claim 1, wherein: the first device is voice controllable; and the second device is a computing device communicatively coupled to the first device. ### Claim 4 (depends on 1) The system of claim 1, the operations further comprising sending, to the first device and the second device, synchronization data configured to synchronize the first portion of the content output by the second device and the second portion of the content output by the first device. ### Claim 5 (depends on 1) The system of claim 1, wherein the first data causes a speaker of the second device to output audio corresponding to the first portion of the content in a time-synchronous manner with output of a visual representation corresponding to the second portion of the content by a display of the first device. ### Claim 6 (depends on 1) The system of claim 1, wherein: the first device is controllable by a third device communicatively coupled to the first device; and the second device is a computing device communicatively coupled to the first device. ### Claim 7 (depends on 1) The system of claim 1, wherein the input data is a touch input received at a touch interface presented on a display of the first device, the display configured to receive touch input, and the operations further comprising: determining a device identifier (ID) of the first device; determining the user profile based at least in part on the device ID; and wherein determining that the second device has been configured to output the first portion of the content while the first device outputs the second portion of the content is based at least in part on the user profile indicating a configuration of the second device. ### Claim 8 (depends on 1) The system of claim 1, wherein the input data is a voice command received at a microphone of a third device, and the operations further comprising: determining a device identifier (ID) of the third device; determining the user profile based at least in part on the device ID; and wherein determining that the second device has been configured to output the first portion of the content while the first device outputs the second portion of the content is based at least in part on the user profile indicating a configuration of the second device. ### Claim 9 (depends on 1) The system of claim 1, wherein the first portion of the content comprises video content and the second portion of the content comprises audio content. ### Claim 10 (depends on 1) The system of claim 1, the operations further comprising: receiving first timestamp information from the first device; receiving second timestamp information from the second device; determining that the first timestamp information differs from the second timestamp information; and sending the first timestamp information to the second device to resynchronize the first portion of the content output by the second device with the second portion of the content output by the first device. ### Claim 11 A method comprising: receiving input data representing a request to output content on a first device associated with a user profile; determining, based at least in part on the user profile, that a second device has been configured to output a first portion of the content; sending, to the second device, first data representing the first portion of the content causing the second device to output the first portion of the content; and sending, to the first device, second data representing the second portion of the content causing the second device to output the first portion of the content in a time-synchronous manner with the second portion of the content output by the first device. ### Claim 12 (depends on 11) The method of claim 11, wherein the second data causes a display of the first device to present a visual representation of the second portion of content in a time-synchronous manner with output of audio corresponding to the first portion of the content by a speaker of the second device. ### Claim 13 (depends on 11) The method of claim 11, wherein: the first device is voice controllable; and the second device is a computing device communicatively coupled to the first device. ### Claim 14 (depends on 11) The method of claim 11, the method further comprising sending, to the first device and the second device, synchronization data configured to synchronize the first portion of the content output by the second device and the second portion of the content output by the first device. ### Claim 15 (depends on 11) The method of claim 11, wherein the first data causes a speaker of the second device to output audio corresponding to the first portion of the content in a time-synchronous manner with output of a visual representation corresponding to the second portion of the content by a display of the first device. ### Claim 16 (depends on 11) The method of claim 11, wherein: the first device is controllable by a third device communicatively coupled to the second device; and the second device is a computing device communicatively coupled to the first device. ### Claim 17 (depends on 11) The method of claim 11, wherein the input data is a touch input received at a touch interface presented on a display of the first device, and the method further comprising: determining a device identifier (ID) of the first device; determining the user profile based at least in part on the device ID; and wherein determining that the second device has been configured to output the first portion of the content while the first device outputs the second portion of the content is based at least in part on the user profile indicating a configuration of the second device. ### Claim 18 (depends on 11) The method of claim 11, wherein the input data is a voice command received at a microphone of a third device, and the method further comprising: determining a device identifier (ID) of the third device; determining the user profile based at least in part on the device ID; and wherein determining that the second device has been configured to output the first portion of the content while the first device outputs the second portion of the content is based at least in part on the user profile indicating a configuration of the second device. ### Claim 19 (depends on 11) The method of claim 11, wherein the first portion of the content comprises audio content and the second portion of the content comprises video content. ### Claim 20 (depends on 11) The method of claim 11, the method further comprising: receiving first timestamp information from the first device; receiving second timestamp information from the second device; determining that the first timestamp information differs from the second timestamp information; and sending the first timestamp information to the second device to resynchronize the first portion of the content output by the second device with the second portion of the content output by the first device.