# US 11,445,236 B2 · Systems and methods for enhanced remote control > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,445,236 B2 - Document ID: US11445236B2 - Status: Granted - Kind: B2 - Application number: 17/177,284 - Filed: 2021-02-17 - Published: 2022-02-03 - Granted: 2022-09-13 - Earliest claimed: 2020-07-31 - Assignees: Arkade, Inc. - Applicants: Arkade, Inc. - Inventors: Joshua Allan Fuchs; Bob Steven Berns; Joel Abraham Kort; Michael Harold Pachter - Web page: https://patentgrep.com/patent/US11445236B2 ## CPC - H04N 21/42222: {Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device} - G08C 17/02: using a radio link - H04N 21/42206: {characterized by hardware details} - H04N 21/42221: {Transmission circuitry, e.g. infrared [IR] or radio frequency [RF]} - H04N 21/42225: {characterized by types of remote control, e.g. universal remote control} - H04N 21/4227: {Providing} Remote input by a user located remotely from the client device, e.g. at work - H04N 21/435: Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream - H04N 21/436: Interfacing a local distribution network, e.g. communicating with another STB {or one or more peripheral devices} inside the home - H04N 21/4363: Adapting the video stream to a specific local network, e.g. a Bluetooth® network - H04N 21/4722: for requesting additional data associated with the content - H04N 21/4728: for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G08C: TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS (fluid pressure transmitting systems F15B; mechanical means for transferring the output of a sensing member into a different variable G01D5/00; mechanical control systems G05G) ## Abstract In some aspects, the disclosure is directed to methods and systems for enhanced remote control of diverse appliances or devices with support for “legacy” devices as well as providing enhanced functionality. In some implementations, the remote control may utilize a multi-device communications scheme and partner with a computing device, such as a smartphone or other such device, to provide added functionality, including network functionality. ## Claims ### Claim 1 A system, comprising: a computing device comprising a processor, a first wireless interface, and a second wireless interface; wherein the processor is configured to: receive, via the first wireless interface from a remote control device, inertial measurement unit (IMU) data, receive, via the second wireless interface from a media appliance, media metadata and a screen capture of a media stream, the media metadata and the screen capture of the media stream transmitted by the media appliance responsive to receipt of a command from the remote control device via a third wireless interface, identify, via the IMU data and screen capture, a selected entity within the screen capture, transmit, to a remote server, an identification of the selected entity and the media metadata, and receive, from the remote server, additional data about the selected entity, the remote server retrieving the additional data based on a search of the identification of the selected entity and the media metadata. ### Claim 2 (depends on 1) The system of claim 1, wherein the processor is further configured to identify a subregion of the screen capture comprising the selected entity based on the IMU data. ### Claim 3 (depends on 2) The system of claim 2, wherein the remote server retrieves the additional data based on a reverse image lookup of the identified subregion of the screen capture and the media metadata. ### Claim 4 (depends on 1) The system of claim 1, wherein the processor is further configured to detect a face in the screen capture. ### Claim 5 (depends on 4) The system of claim 4, wherein the remote server retrieves the additional data based on a reverse image lookup of the detected face and the media metadata. ### Claim 6 (depends on 1) The system of claim 1, wherein the first wireless interface communicates with the remote control device via a Bluetooth communication protocol and the second wireless interface communicates with the media appliance via a WiFi communication protocol. ### Claim 7 (depends on 1) The system of claim 1, wherein the processor is configured to receive additional data about the selected entity by receiving information about products that are visible in the screen capture from an online retailer. ### Claim 8 (depends on 1) The system of claim 1, wherein the third wireless interface communicates with the media appliance via an infrared communication protocol. ### Claim 9 (depends on 1) The system of claim 1, wherein the processor is configured to receive the media metadata by receiving an identification of a show title and a timestamp. ### Claim 10 (depends on 1) The system of claim 1, wherein the processor is further configured to display the additional data about the selected entity on a display of the computing device or a display of the media appliance. ### Claim 11 A method, comprising: receiving, by a computing device via a first wireless interface from a remote control device, inertial measurement unit (IMU) data; receiving, by the computing device via a second first wireless interface from a media appliance, media metadata and a screen capture of a media stream, the media metadata and the screen capture of the media stream transmitted by the media appliance responsive to receipt of a command from the remote control device via a third wireless interface; identifying, by the computing device via the IMU data and screen capture, a selected entity within the screen capture; transmitting, by the computing device to a remote server, an identification of the selected entity and the media metadata; and receiving, by the computing device from the remote server, additional data about the selected entity, the remote server retrieving the additional data based on a search of the identification of the selected entity and the media metadata. ### Claim 12 (depends on 11) The method of claim 11, further comprising identifying, by the computing device, a subregion of the screen capture comprising the selected entity based on the IMU data. ### Claim 13 (depends on 12) The method of claim 12, wherein the remote server retrieves the additional data based on a reverse image lookup of the identified subregion of the screen capture and the media metadata. ### Claim 14 (depends on 11) The method of claim 11, further comprising detecting, by the computing device, a face in the screen capture. ### Claim 15 (depends on 14) The method of claim 14, wherein the remote server retrieves the additional data based on a reverse image lookup of the detected face and the media metadata. ### Claim 16 (depends on 11) The method of claim 11, wherein the first wireless interface communicates with the remote control device via a Bluetooth communication protocol and the second wireless interface communicates with the media appliance via a WiFi communication protocol. ### Claim 17 (depends on 11) The method of claim 11, wherein receiving additional data about the selected entity comprises receiving information about products that are visible in the screen capture from an online retailer. ### Claim 18 (depends on 11) The method of claim 11, wherein the third wireless interface communicates with the media appliance via an infrared communication protocol.