# US 2020/0404371 A1 · Use of Steganographically-Encoded Data as Basis to Control Dynamic Content Modification as to at Least One Modifiable-Content Segment Identified Based on Fingerprint Analysis > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0404371 A1 - Document ID: US20200404371A1 - Status: Published application - Kind: A1 - Application number: 15/929,225 - Filed: 2020-02-14 - Published: 2020-12-24 - Earliest claimed: 2019-06-18 - Assignees: The Nielsen Company (US), LLC - Applicants: The Nielsen Company (US), LLC - Inventors: Shashank Merchant; Do Phuoc Hoang Tuong Lan - Web page: https://patentgrep.com/patent/US20200404371A1 ## CPC - H04N 21/44016: {involving splicing one content stream with another content stream, e.g. for substituting a video clip} - H04N 21/440245: {the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment} - G06F 16/783: using metadata automatically derived from the content - H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/458: Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules {; time-related management operations (arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10)} - H04N 21/4622: {Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet (web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859)} - H04N 21/4627: Rights management {associated to the content (security in data switching network management H04L41/28; security management or policies for network security H04L63/20; access security in wireless networks H04W12/08)} - H04N 21/4852: {for modifying audio parameters, e.g. switching between mono and stereo} - H04N 21/4854: {for modifying image parameters, e.g. image brightness, contrast} - H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)} - H04N 21/8358: involving watermark {(protecting executable software by watermarking G06F21/16; image watermarking in general G06T1/0021; watermarks inserted in still images for transmission purposes H04N1/32144; inserting watermarks during video coding H04N19/467)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The disclosure provides for use of steganographically-encoded data as a basis to control content modification. A server could determine through a fingerprint comparison that a portion of media content on a channel being rendered by a content presentation device matches at least one modifiable-content segment, such as at least one replaceable ad. The server could then use watermark data encoded in the media content of the channel as a basis to determine an actual identity of a modifiable-content segment present on the channel, so as to then control content modification. ## Claims ### Claim 1 A method comprising: determining based on a fingerprint comparison that a portion of media content on a channel being rendered by a content presentation device matches at least one modifiable-content segment; and responsive to the determining, using watermark data encoded on the channel as a basis to control modification of the media content. ### Claim 2 (depends on 1) The method of claim 1, wherein each of the at least one modifiable-content segment is a replacement ad. ### Claim 3 (depends on 1) The method of claim 1, wherein using the watermark data encoded on the channel as a basis to control the modification of the media content comprises using the watermark data encoded on the channel as a basis to control whether to allow the modification to occur. ### Claim 4 (depends on 1) The method of claim 1, wherein determining based on the fingerprint comparison that the portion of media content on the channel being rendered by a content presentation device matches at least one modifiable-content segment comprises determining based on the fingerprint comparison that the portion of the media content matches a particular modifiable-content segment, and wherein using the watermark data encoded on the channel as a basis to control modification of the media content comprises (i) ascertaining from the watermark data an actual identity of a modifiable-content segment present on the channel and making a determination of whether the actually identified modifiable-content segment is the particular modifiable-content segment, (ii) if the determination is affirmative, then, based at least in part on the determination, allowing the modification of the media content to proceed, and (iii) if the determination is negative, then, based at least in part on the determination, causing abandonment of the modification of the media content. ### Claim 5 (depends on 4) The method of claim 4, wherein the method is carried out by at least a server that is in network communication with the content presentation device, and wherein using the watermark data encoded on the channel as a basis to control modification of the media content is carried out after the server has directed the content presentation device to carry out the modification. ### Claim 6 (depends on 1) The method of claim 1, wherein determining based on the fingerprint comparison that the portion of media content on the channel being rendered by a content presentation device matches at least one modifiable-content segment comprises determining based on the fingerprint comparison that the portion of the media content matches multiple modifiable-content segments, and wherein using the watermark data encoded on the channel as a basis to control modification of the media content comprises (i) ascertaining from the watermark data an actual identity of a modifiable-content segment present on the channel and (ii) using the ascertained actual identity as a basis to disambiguate so as to determine which of the multiple modifiable-content segments is the modifiable-content segment present on the channel. ### Claim 7 (depends on 6) The method of claim 6, wherein the method is carried out by at least a server that is in network communication with the content presentation device, and wherein using the watermark data encoded on the channel as a basis to control modification of the media content is carried out after the server has directed the content presentation device to carry out the modification. ### Claim 8 (depends on 1) The method of claim 1, wherein the fingerprint comparison is video based and the watermark data is audio based. ### Claim 9 A system comprising: a network communication interface; a processing unit; non-transitory data storage; and program instructions stored in the non-transitory data storage and executable by the processing unit to carry out operations including: determining based on a fingerprint comparison that a portion of media content on a channel being rendered by a content presentation device matches at least one modifiable-content segment, and responsive to at least the determining, using watermark data encoded on the channel as a basis to control modification of the media content. ### Claim 10 (depends on 9) The system of claim 9, wherein each of the at least one modifiable-content segment is a replacement ad. ### Claim 11 (depends on 9) The system of claim 9, wherein using the watermark data encoded on the channel as a basis to control the modification of the media content comprises using the watermark data encoded on the channel as a basis to control whether to allow the modification to occur. ### Claim 12 (depends on 9) The system of claim 9, wherein determining based on the fingerprint comparison that the portion of media content on the channel being rendered by a content presentation device matches at least one modifiable-content segment comprises determining based on the fingerprint comparison that the portion of the media content matches a particular modifiable-content segment, and wherein using the watermark data encoded on the channel as a basis to control modification of the media content comprises (i) ascertaining from the watermark data an actual identity of a modifiable-content segment present on the channel and making a determination of whether the actually identified modifiable-content segment is the particular modifiable-content segment, (ii) if the determination is affirmative, then, based at least in part on the determination, allowing the modification of the media content to proceed, and (iii) if the determination is negative, then, based at least in part on the determination, causing abandonment of the modification of the media content. ### Claim 13 (depends on 12) The system of claim 12, wherein the system is separate from the content presentation device, and wherein using the watermark data encoded on the channel as a basis to control modification of the media content is carried out after the system has directed the content presentation device to carry out the modification. ### Claim 14 (depends on 9) The system of claim 9, wherein determining based on the fingerprint comparison that the portion of media content on the channel being rendered by a content presentation device matches at least one modifiable-content segment comprises determining based on the fingerprint comparison that the portion of the media content matches multiple modifiable-content segments, and wherein using the watermark data encoded on the channel as a basis to control modification of the media content comprises (i) ascertaining from the watermark data an actual identity of a modifiable-content segment present on the channel and (ii) using the ascertained actual identity as a basis to disambiguate so as to determine which of the multiple modifiable-content segments is the modifiable-content segment present on the channel. ### Claim 15 (depends on 14) The system of claim 14, wherein the system is separate from the content presentation device, and wherein using the watermark data encoded on the channel as a basis to control modification of the media content is carried out after the system has directed the content presentation device to carry out the modification. ### Claim 16 (depends on 9) The system of claim 9, wherein the fingerprint comparison is video based and the watermark data is audio based. ### Claim 17 A method comprising: starting by a content presentation device to prepare for carrying out content modification with respect to a portion of media content on a channel being rendered by the content presentation device, wherein the starting is responsive to signaling from a server based on the server having determined through a fingerprint comparison that the portion of the media content is a particular modifiable-content segment; and before starting by the content presentation device to carry out the content modification, abandoning by the content presentation device the content modification, wherein the abandoning is responsive to further signaling from the server based on the server having determined based on watermark data encoded in media content of the channel that the portion of the media content is not the particular modifiable-content segment. ### Claim 18 (depends on 17) The method of claim 17, wherein the starting to prepare for carrying out the content modification comprises an action selected from the group consisting of (i) receiving a directive for the content presentation device to carry out the content modification, (ii) querying storage to obtain modified content specifically for presentation in place of the particular modifiable-content segment, and (iii) setting a trigger for beginning to play out of the modified content in place of the particular modifiable-content segment. ### Claim 19 (depends on 17) The method of claim 17, wherein the particular modifiable-content segment is a replaceable ad. ### Claim 20 (depends on 17) The method of claim 17, wherein the fingerprint comparison is video based and the watermark data is audio based.