# US 10,771,848 B1 · Actionable contents of interest > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,771,848 B1 - Document ID: US10771848B1 - Status: Granted - Kind: B1 - Application number: 16/299,938 - Filed: 2019-03-12 - Granted: 2020-09-08 - Earliest claimed: 2019-01-07 - Assignees: Alphonso Inc. - Applicants: Alphonso Inc. - Inventors: Manish Gupta; Lampros Kalampoukas; Raghavendra Koushik; Subbhaash S - Web page: https://patentgrep.com/patent/US10771848B1 ## CPC - H04N 21/462: Content or additional data management e.g. creating a master electronic programme guide from data received from the Internet and a Head-end or controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities - H04N 21/4756: {for rating content, e.g. scoring a recommended movie} - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/25891: {being end-user preferences (retrieval of video data in a video database based on user preferences G06F16/739; arrangements for recognizing users' preferences H04H60/46; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N 21/44: Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/4532: {involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N 21/482: End-user interface for programme selection - H04N 21/6582: {Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38)} - H04N 21/84: Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} - H04N 21/8545: for generating interactive applications - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Methods and systems are provided to create a media consumer-personalized database table of selectable actions related to media content of interest to a media consumer that is being displayed on a media device. The media content of interest is identified by the media consumer by recording an indication of the media content of interest via a signal received from a user input device when the media content of interest is being displayed or presented. The database table of selectable actions is then presented to the media consumer who can initiate one of the actions. ## Claims ### Claim 1 An automated method of creating a media consumer-personalized database table of selectable actions related to media content of interest to a media consumer that is being displayed on a media device, and using the database table of selectable actions to initiate an action, wherein a media content database maintains (i) an identifier of media content that is broadcast or streamed to a plurality of media devices, and (ii) metadata associated with the media content, the method comprising: (a) recording, by the media device, an indication of media content of interest to the media consumer via a pre-designated input received by the media device from a user input device when the media content of interest is being displayed; (b) creating a fingerprint, by a fingerprint processor, for a segment of each indicated media content of interest, and electronically communicating the fingerprint to a remote server; (c) performing automated content recognition (ACR) in the remote server on the fingerprints, the remote server thereby identifying the media content for each of the media consumer's indicated media content of interest by its respective identifier; (d) populating, by the remote server that is in electronic communication with the media content database, the database table of selectable actions with at least the following items for each indicated media content of interest: (i) the metadata associated with the media content of interest, and (ii) selectable actions that can be taken using the metadata related to the media content of interest, wherein the selectable actions are based on the metadata related to the media content of interest, thereby allowing different selectable actions to be taken for different media content of interest, and wherein the database table of selectable actions is populated with the metadata and associated selectable actions for a plurality of different media content of interest; (e) displaying in a single display screen, on the media device, the populated database table of selectable actions for the plurality of different media content of interest, wherein the single display screen shows the plurality of different media content of interest, and their respective selectable actions; and (f) initiating, by the media device, at least one of the selectable actions upon receipt by the media consumer or another entity of a request to take one of the selectable actions, wherein at least some of the selectable actions require the media device to electronically communicate with an entity external to the media device to perform the selectable actions, and wherein steps (e) and (f) occur at a later point in time than step (a). ### Claim 2 (depends on 1) The method of claim 1 wherein the media content database further maintains: (iii) media content attribution data that identifies an attribution entity to receive attribution credit for particular media content, the method further comprising: (g) providing attribution credit to the respective attribution entity, using the media content attribution data, upon initiation of a selectable action for the particular media content. ### Claim 3 (depends on 1) The method of claim 1 wherein the selectable actions use the metadata to initiate the request. ### Claim 4 (depends on 1) The method of claim 1 wherein the media device includes an operating system, and wherein at least steps (a), (e) and (f) are performed using the operating system of the media device. ### Claim 5 (depends on 1) The method of claim 1 wherein the media device is one of a smart television, and a smart multi-media player device. ### Claim 6 (depends on 1) The method of claim 1 wherein the user input device is a remote control that receives input via at least one of physical touch and voice commands. ### Claim 7 (depends on 1) The method of claim 1 wherein the fingerprints are one or more of audio and video fingerprints. ### Claim 8 (depends on 1) The method of claim 1 wherein the selectable actions depend, in part, upon the type of media content. ### Claim 9 (depends on 1) The method of claim 1 wherein the media content database further maintains (iii) the selectable actions that can be taken using the metadata related to the media content, and wherein the selectable actions in the database table of selectable actions are obtained from the selectable actions in the media content database. ### Claim 10 (depends on 1) The method of claim 1 wherein the selectable actions in the database table of selectable actions are created on-the-fly from at least the metadata associated with the media content. ### Claim 11 (depends on 1) The method of claim 1 wherein the fingerprint processor is in the media device, and wherein in step (b), the created fingerprint for the segment of each indicated media content of interest is electronically communicated by the media device to the remote server. ### Claim 12 (depends on 1) The method of claim 1 wherein the fingerprint processor is in the remote server, and the method further comprises: (g) capturing, by the media device, a media segment of each indicated media content of interest, and electronically transmitting the segment to the remote server for creation of the fingerprint by the fingerprint processor in the remote server. ### Claim 13 (depends on 1) The method of claim 1 further comprising: (g) capturing, by the media device, electronic program guide (EPG) data regarding the indicated media content of interest, and electronically transmit the EPG data to the remote server, wherein step (c) further comprises, the remote server also using the EPG data to identify the media content. ### Claim 14 (depends on 1) The method of claim 1 further comprising: (g) capturing, by the media device, electronic program guide (EPG) data regarding the indicated media content of interest, and electronically transmitting the EPG data to the remote server, wherein step (c) further comprises, the remote server using the EPG data to identify the media content when the media content cannot be properly identified using ACR. ### Claim 15 An automated system for creating a media consumer-personalized database table of selectable actions related to media content of interest to a media consumer that is being displayed on a media device, and using the database table of selectable actions to initiate an action, wherein a media content database maintains (i) an identifier of media content that is broadcast or streamed to a plurality of media devices, and (ii) metadata associated with the media content, the system comprising: (a) program code in the media device configured to record an indication of media content of interest to the media consumer via a pre-designated input received by the media device from a user input device when the media content of interest is being displayed; (b) a fingerprint processor configured to create a fingerprint for a segment of each indicated media content of interest; and (c) a remote server that is in electronic communication with the media content database, the remote server configured to: (i) perform automated content recognition (ACR) on the fingerprints, the remote server thereby identifying the media content for each of the media consumer's indicated media content of interest by its respective identifier, and (ii) populate the database table of selectable actions with at least the following items for each indicated media content of interest: (A) the metadata associated with the media content of interest, and (B) selectable actions that can be taken using the metadata related to the media content of interest, wherein the selectable actions are based on the metadata related to the media content of interest, thereby allowing different selectable actions to be taken for different media content of interest, and wherein the database table of selectable actions is populated with the metadata and associated selectable actions for a plurality of different media content of interest, the program code in the media device being further configured to: (i) display, in a single display screen, on the media device, the populated database table of selectable actions for the plurality of different media content of interest, wherein the single display screen shows the plurality of different media content of interest, and their respective selectable actions, (ii) initiate at least one of the selectable actions upon receipt by the media consumer or another entity of a request to take one of the selectable actions, wherein at least some of the selectable actions require the media device to electronically communicate with an entity external to the media device to perform the selectable actions, and wherein the display of the populated database table of selectable actions and the initiation of at least one of the selectable actions occur at a later point in time than when the media content of interest to the media consumer is being displayed. ### Claim 16 (depends on 15) The system of claim 15 wherein the media content database further maintains: (iii) media content attribution data that identifies an attribution entity to receive attribution credit for particular media content, the system further comprising: (d) an attribution model configured to provide attribution credit to the respective attribution entity, using the media content attribution data, upon initiation of a selectable action for the particular media content. ### Claim 17 (depends on 15) The system of claim 15 wherein the selectable actions use the metadata to initiate the request. ### Claim 18 (depends on 15) The system of claim 15 wherein the media device includes an operating system, and wherein the program code executes in the operating system. ### Claim 19 (depends on 15) The system of claim 15 wherein the media device is one of a smart television, and a smart multi-media player device. ### Claim 20 (depends on 15) The system of claim 15 wherein the user input device is a remote control that receives input via at least one of physical touch and voice commands. ### Claim 21 (depends on 15) The system of claim 15 wherein the fingerprints are one or more of audio and video fingerprints. ### Claim 22 (depends on 15) The system of claim 15 wherein the selectable actions depend, in part, upon the type of media content. ### Claim 23 (depends on 15) The system of claim 15 wherein the media content database further maintains (iii) the selectable actions that can be taken using the metadata related to the media content, and wherein the selectable actions in the database table of selectable actions are obtained from the selectable actions in the media content database. ### Claim 24 (depends on 15) The system of claim 15 wherein the selectable actions in the database table of selectable actions are created on-the-fly from at least the metadata associated with the media content. ### Claim 25 (depends on 15) The system of claim 15 wherein the fingerprint processor is part of the program code of the media device, and the program code of the media device is further configured to: (iii) electronically communicate the fingerprint to the remote server. ### Claim 26 (depends on 15) The system of claim 15 wherein the fingerprint processor is in the remote server, and the program code of the media device is further configured to: (iii) capture a media segment of each indicated media content of interest, and electronically transmit the segment to the remote server for creation of the fingerprint by the fingerprint processor in the remote server. ### Claim 27 (depends on 15) The system of claim 15 wherein the program code of the media device is further configured to: (iii) capture electronic program guide (EPG) data regarding the indicated media content of interest, and electronically transmit the EPG data to the remote server, wherein the remote server also uses the EPG data to identify the media content. ### Claim 28 (depends on 15) The system of claim 15 wherein the program code of the media device is further configured to: (iii) capture electronic program guide (EPG) data regarding the indicated media content of interest, and electronically transmit the EPG data to the remote server, wherein the remote server uses the EPG data to identify the media content when the media content cannot be properly identified using ACR.