# US 11,445,258 B2 · Method and system for presenting additional content at a media system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,445,258 B2 - Document ID: US11445258B2 - Status: Granted - Kind: B2 - Application number: 16/835,215 - Filed: 2020-03-30 - Published: 2020-07-16 - Granted: 2022-09-13 - Earliest claimed: 2010-11-01 - Assignees: Roku, Inc. - Applicants: Roku, Inc. - Inventors: Donald F. Gordon - Web page: https://patentgrep.com/patent/US11445258B2 ## CPC - H04N 21/4722: for requesting additional data associated with the content - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/23892: {involving embedding information at multiplex stream level, e.g. embedding a watermark at packet level} - H04N 21/254: Management at additional data server, e.g. shopping server, rights management server {(arrangements for maintenance or administration in data networks H04L41/00; network services using third party service providers H04L67/53)} - H04N 21/278: Content descriptor database or directory service for end-user access {(details of content or meta data based information retrieval of video data in video databases G06F16/739)} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/4402: involving reformatting operations of video signals for household redistribution, storage or real-time display {(details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764)} - H04N 21/44213: {Monitoring of end-user related data (arrangements for monitoring the users' behaviour or opinions in broadcast systems H04H60/33)} - H04N 21/6581: {Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application} - H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)} - H04N 21/8133: {specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video programme} - H04N 21/8358: involving watermark {(protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A media system, receives a received sequence of media content, for presentation at the media system and generates a comparison fingerprint of the received sequence of media content. The comparison fingerprint is for comparison with a plurality of reference fingerprints so as to identify the received sequence of media content. The media system sends a request for identification of additional content to a server system. The request is based at least in part on the comparison fingerprint. The media system receives a response to the request, including information enabling additional content to be selected for display at the media system based at least in part on the identification of the received sequence of media content, and presents a displayed sequence of media content that includes at least a portion of the received sequence of media content and at least a portion of the additional content. ## Claims ### Claim 1 A method comprising: receiving and presenting, by a client device, a sequence of media content; generating, by the client device, first comparison fingerprint data representing the sequence of media content for comparison with a plurality of sets of reference fingerprint data; receiving, by the client device, an indication that the first comparison fingerprint data matches multiple sets of reference fingerprint data of the plurality of sets of reference fingerprint data, wherein each of the multiple sets of reference fingerprint data correspond to different sequences of reference media content, and wherein the received indication comprises multiple sets of additional reference fingerprint data, each set of the multiple sets of additional reference fingerprint data corresponding to a respective one of the different sequences of reference media content; generating, by the client device, second comparison fingerprint data representing the sequence of media content; and comparing, by the client device, the generated second comparison fingerprint data with the received multiple sets of additional reference fingerprint data. ### Claim 2 (depends on 1) The method of claim 1, wherein the multiple sets of additional reference fingerprint data corresponding to different sequences of reference media content include reference fingerprint data corresponding to: (i) a first sequence of reference media content that matches the sequence of media content being presented by the client device and (ii) a second sequence of reference media content that does not match the sequence of media content being presented by the client device. ### Claim 3 (depends on 2) The method of claim 2, further comprising: determining, by the client device, that the second comparison fingerprint data matches a given one of the received multiple sets of additional fingerprint data corresponding to the first sequence of reference media content; and based on the determining, the client device identifying the sequence of media content being presented by the client device as the first sequence of reference media content. ### Claim 4 (depends on 3) The method of claim 3, wherein determining that the second comparison fingerprint data matches the given one of the received multiple sets of additional fingerprint data comprises determining that the second comparison fingerprint data matches only the given one of the received multiple sets of additional fingerprint data corresponding to the first sequence of reference media content and that the second comparison fingerprint data does not match another of the received multiple sets of additional fingerprint data corresponding to any other sequence of the different sequences of reference media content. ### Claim 5 (depends on 3) The method of claim 3, further comprising, subsequent, to the client device identifying the sequence of media content being presented by the client device as the first sequence of reference media content: receiving and presenting, by the client device, a second sequence of media content; receiving second additional reference fingerprint data corresponding to the first sequence of reference media content; and generating third comparison fingerprint data representing the second sequence of media content for comparison with the second additional reference fingerprint data so as to identify whether the second sequence of media content is a continuation of the first sequence of reference media content. ### Claim 6 (depends on 1) The method of claim 1, further comprising, based on the comparing, the client device sending to a server a request for supplemental media content. ### Claim 7 (depends on 6) The method of claim 6, wherein the request includes an indication of a result of the comparing. ### Claim 8 (depends on 7) The method of claim 7, wherein the request includes an indication of a matching one of the multiple sets of additional reference fingerprint data. ### Claim 9 A non-transitory computer-readable storage medium having stored thereon program instructions that, when executed by one or more processors of a client device, cause the client device to perform operations comprising: receiving and presenting a sequence of media content; generating first comparison fingerprint data representing the sequence of media content for comparison with a plurality of sets of reference fingerprint data; receiving an indication that the first comparison fingerprint data matches multiple sets of reference fingerprint data of the plurality of sets of reference fingerprint data, wherein each of the multiple sets of reference fingerprint data correspond to different sequences of reference media content, and wherein the received indication comprises multiple sets of additional reference fingerprint data, each set of the multiple sets of additional, reference fingerprint data corresponding to a respective one of the different sequences of reference media content; generating second comparison fingerprint data representing the sequence of media content; and comparing the generated second comparison fingerprint data with the received multiple sets of additional reference fingerprint data. ### Claim 10 (depends on 9) The non-transitory computer-readable storage medium of claim 9, wherein the multiple sets of additional reference fingerprint data corresponding to different sequences of reference media content include reference fingerprint data corresponding to: (i) a first sequence of reference media content that matches the sequence of media content being presented by the client device and (ii) a second sequence of reference media content that does not match the sequence of media content being presented by the client device. ### Claim 11 (depends on 10) The non-transitory computer-readable storage medium of claim 10, the operations further comprising: determining that the second comparison fingerprint data matches a given one of the received multiple sets of additional fingerprint data corresponding to the first sequence of reference media content; and based on the determining, identifying the sequence of media content being presented by the client device as the first sequence of reference media content. ### Claim 12 (depends on 11) The non-transitory computer-readable storage medium of claim 11, wherein determining that the second comparison fingerprint data matches the given one of the received multiple sets of additional fingerprint data comprises determining that the second comparison fingerprint data matches only the given one of the received multiple sets of additional fingerprint data corresponding to the first sequence of reference media content and that the second comparison fingerprint data does not match another of the received multiple sets of additional fingerprint data corresponding to any other sequence of the different sequences of reference media content. ### Claim 13 (depends on 11) The non-transitory computer-readable storage medium of claim 11, the operations further comprising, subsequent to the client device identifying the sequence of media content being presented by the client device as the first sequence of reference media content: receiving and presenting a second sequence of media content; receiving second additional reference fingerprint data corresponding to the first sequence of reference media content; and generating third comparison fingerprint data representing the second sequence of media content for comparison with the second additional reference fingerprint data so as to identify whether the second sequence of media content is a continuation of the first sequence of reference media content. ### Claim 14 (depends on 9) The non-transitory computer-readable storage medium of claim 9, the operations further comprising, based on the comparing, sending to a server a request for supplemental media content, wherein the request includes an indication of a matching one of the multiple sets of additional reference fingerprint data. ### Claim 15 A client device comprising one or more processors and a non-transitory computer-readable storage medium having stored thereon program instructions that, when executed by the one or more processors, cause the client device to perform operations comprising: receiving and presenting a sequence of media content; generating first comparison fingerprint data representing the sequence of media content for comparison with a plurality of sets of reference fingerprint data; receiving an indication that the first comparison fingerprint data matches multiple sets of reference fingerprint data of the plurality of sets of reference fingerprint data, wherein each of the multiple sets of reference fingerprint data correspond to different sequences of reference media content, and wherein the received indication comprises multiple sets of additional, reference fingerprint data, each set of the multiple sets of additional reference fingerprint data corresponding to a respective one of the different sequences of reference media content; generating second comparison fingerprint data representing the sequence of media content; and comparing the generated second comparison fingerprint data with the received multiple sets of additional reference fingerprint data. ### Claim 16 (depends on 15) The client device of claim 15, wherein the multiple sets of additional reference fingerprint data corresponding to different sequences of reference media content include reference fingerprint data corresponding to: (i) a first sequence of reference media content that matches the sequence of media content being presented by the client device and (ii) a second sequence of reference media content that does not match the sequence of media content being presented by the client device. ### Claim 17 (depends on 16) The client device of claim 16, the operations further comprising: determining that the second comparison fingerprint data matches a given one of the received multiple sets of additional fingerprint data corresponding to the first sequence of reference media content; and based on the determining, identifying the sequence of media content being presented by the client device as the first sequence of reference media content. ### Claim 18 (depends on 17) The client device of claim 17, wherein determining that the second comparison fingerprint data matches the given one of the received multiple sets of additional fingerprint data comprises determining that the second comparison fingerprint data matches only the given one of the received multiple sets of additional fingerprint data corresponding to the first sequence of reference media content and that the second comparison fingerprint data does not match another of the received multiple sets of additional fingerprint data corresponding to any other sequence of the different sequences of reference media content. ### Claim 19 (depends on 17) The client device of claim 17, the operations further comprising, subsequent to the client device identifying the sequence of media content being presented by the client device as the first sequence of reference media content: receiving and presenting a second sequence of media content; receiving second additional reference fingerprint data corresponding to the first sequence of reference media content; and generating third comparison fingerprint data representing the second sequence of media content for comparison with the second additional reference fingerprint data so as to identify whether the second sequence of media content is a continuation of the first sequence of reference media content. ### Claim 20 (depends on 16) The client device of claim 16, the operations further comprising, based on the comparing, sending to a server a request for supplemental media content, wherein the request includes an indication of a matching one of the multiple sets of additional reference fingerprint data.