# US 12,225,263 B2 · Systems and methods for operating a set top box > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,225,263 B2 - Document ID: US12225263B2 - Status: Granted - Kind: B2 - Application number: 18/233,608 - Filed: 2023-08-14 - Published: 2023-11-30 - Granted: 2025-02-11 - Earliest claimed: 2012-09-19 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Vincent Dureau; Nathan Leslie Sandland - Web page: https://patentgrep.com/patent/US12225263B2 ## CPC - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 5/50: Tuning indicators; Automatic tuning control - H04N 21/41407: {embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57)} - H04N 21/4227: {Providing} Remote input by a user located remotely from the client device, e.g. at work - H04N 21/4325: {by playing back content from the storage medium (reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79)} - H04N 21/4334: {Recording operations (recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27)} - H04N 21/4532: {involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N 21/47202: {for requesting content on demand, e.g. video on demand} - H04N 21/47214: {for content reservation or setting reminders; for requesting event notification, e.g. of sport results or stock market (stock exchange data over packet-switching network H04L12/1804; push services over packet-switching network H04L12/1859; notification of incoming messages in packet switching networks H04L51/224)} - H04N 21/482: End-user interface for programme selection - H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Features of various implementations are used to control operation of a second electronic device from a first electronic device using indirect control methods and optical character recognition. In some implementations, communication between a primary set-top box and a supplemental set-top box is configured in order to avoid the need for one or more provider-specific APIs. For example, one aspect of the disclosure is a method of identifying commands that can be used to control one or more features of a primary set-top box (e.g., DVR or VOD features) by a supplemental set-top box issuing commands to the primary set-top box via a wireless (e.g., IR) transmitter, and then using optical character recognition to identify the outcomes of those commands as displayed on a primary TV display or secondary display. ## Claims ### Claim 1 A computer-implemented method comprising: at a first electronic device coupled to a display and a second electronic device, the first electronic device including one or more processors and memory: transmitting a first command sequence to the second electronic device; determining, based on captured display data provided by the second electronic device, a first state of the second electronic device by comparing a fingerprint of the captured display data to one or more fingerprints corresponding to a plurality of expected display responses; based on the first state of the second electronic device, transmitting a second command sequence to the second electronic device, wherein the second command sequence controls a feature provided by the second electronic device; and capturing audio content provided by the second electronic device, wherein the first state of the second electronic device is determined by comparing an audio fingerprint generated from the captured audio content to one or more fingerprints for expected content. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the method further comprises capturing the display data provided by the second electronic device in response to transmitting the first command sequence to the second electronic device. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein the method further comprises generating a fingerprint of displayed user interface elements from the captured display data. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the plurality of expected display responses comprise at least one of bounding box size coordinates, channel number, and text overlays. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the second electronic device provides a plurality of features and wherein the method further comprises selecting the second command sequence from a database of command sequences based on a device type of the second electronic device and the plurality of features of the second electronic device. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the first electronic device further includes a microphone and wherein the method further comprises capturing, via the microphone, outputted audio content provided by the second electronic device. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the first command sequence and the second command sequence are trial command sequences that are not known to be correct for the second electronic device. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein the second command sequence is transmitted to the second electronic device in response to determining that the first command sequence was not performed correctly by the second electronic device. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the method further comprises identifying a command set that includes the second command sequence as a correct command set for the second electronic device in response to determining that the second command sequence was performed correctly by the second electronic device. ### Claim 10 An electronic device comprising: a memory; and a hardware processor that, when executing computer-executable instructions stored in the memory: transmits a first command sequence to a second electronic device, wherein the electronic device coupled to a display and the second electronic device; determines, based on captured display data provided by the second electronic device, a first state of the second electronic device by comparing a fingerprint of the captured display data to one or more fingerprints corresponding to a plurality of expected display responses; based on the first state of the second electronic device, transmits a second command sequence to the second electronic device, wherein the second command sequence controls a feature provided by the second electronic device; and captures audio content provided by the second electronic device, wherein the first state of the second electronic device is determined by comparing an audio fingerprint generated from the captured audio content to one or more fingerprints for expected content. ### Claim 11 (depends on 10) The electronic device of claim 10, wherein the hardware processor further captures the display data provided by the second electronic device in response to transmitting the first command sequence to the second electronic device. ### Claim 12 (depends on 11) The electronic device of claim 11, wherein the hardware processor further generates a fingerprint of displayed user interface elements from the captured display data. ### Claim 13 (depends on 10) The electronic device of claim 10, wherein the plurality of expected display responses comprise at least one of bounding box size coordinates, channel number, and text overlays. ### Claim 14 (depends on 10) The electronic device of claim 10, wherein the second electronic device provides a plurality of features and wherein the method further comprises selecting the second command sequence from a database of command sequences based on a device type of the second electronic device and the plurality of features of the second electronic device. ### Claim 15 (depends on 10) The electronic device of claim 10, wherein the electronic device further includes a microphone and wherein the method further comprises capturing, via the microphone, outputted audio content provided by the second electronic device. ### Claim 16 (depends on 10) The electronic device of claim 10, wherein the first command sequence and the second command sequence are trial command sequences that are not known to be correct for the second electronic device. ### Claim 17 (depends on 16) The electronic device of claim 16, wherein the second command sequence is transmitted to the second electronic device in response to determining that the first command sequence was not performed correctly by the second electronic device. ### Claim 18 (depends on 10) The electronic device of claim 10, wherein the hardware processor further identifies a command set that includes the second command sequence as a correct command set for the second electronic device in response to determining that the second command sequence was performed correctly by the second electronic device. ### Claim 19 A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor of a first electronic device, cause the processor to perform a method, the method comprising: transmitting a first command sequence to a second electronic device, wherein the first electronic device is coupled to a display and a second electronic device; determining, based on captured display data provided by the second electronic device, a first state of the second electronic device by comparing a fingerprint of the captured display data to one or more fingerprints corresponding to a plurality of expected display responses; based on the first state of the second electronic device, transmitting a second command sequence to the second electronic device, wherein the second command sequence controls a feature provided by the second electronic device; and capturing audio content provided by the second electronic device, wherein the first state of the second electronic device is determined by comparing an audio fingerprint generated from the captured audio content to one or more fingerprints for expected content.