# US 2021/0084091 A1 · Use of In-Band Metadata as Basis to Access Reference Fingerprints to Facilitate Content-Related Action > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0084091 A1 - Document ID: US20210084091A1 - Status: Published application - Kind: A1 - Application number: 15/929,210 - Filed: 2020-01-28 - Published: 2021-03-18 - Earliest claimed: 2019-09-13 - Assignees: The Nielsen Company (US), LLC - Applicants: The Nielsen Company (US), LLC - Inventors: Steven M. Cormie; Brian Jentz - Web page: https://patentgrep.com/patent/US20210084091A1 ## CPC - H04L 65/1089: by adding media; by removing media - H04L 65/61: for supporting one-way streaming services, e.g. Internet radio - H04L 65/612: for unicast - H04L 65/764: {at the destination (reformatting of additional data in video clients H04N21/4355)} - H04L 69/14: Multichannel or multilink protocols - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract As a media client receives a media stream for presentation, the media client could receive, in-band with the media stream, metadata identifying the media stream, such as a media-stream identifier encoded as a watermark in the media stream or carried together with the media stream in a packet-based transport stream, among other possibilities. And while the media client is processing the media stream for presentation, the media client could then convey the received metadata out-of-band to a server to enable the server to readily obtain reference fingerprints representing the media stream and to thereby facilitate, based on the reference fingerprints, a content-related action such as dynamic content modification for instance. ## Claims ### Claim 1 A method comprising: receiving by a media client a media stream, and receiving by the media client, as in-band metadata with the media stream, an identifier of the media stream; processing by the media client the media stream for presentation; and while processing by the media client the media stream for presentation, transmitting by the media client to a server, in out-of-band signaling, the identifier of the media stream received by the media client as in-band metadata with the media stream, to enable the server to obtain reference fingerprints representing the media stream without a need for the server to search for reference fingerprints that match query fingerprints representing the media stream being processed by the media client. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving by the media client from the server a directive for the media client to perform a content-related action at a particular time in the media stream, wherein the particular time is established based on the obtained reference fingerprints. ### Claim 3 (depends on 2) The method of claim 2, wherein the content-related action comprises dynamic ad insertion. ### Claim 4 (depends on 1) The method of claim 1, wherein receiving by the media client the identifier as in-band metadata with the media stream comprises receiving by the media client the identifier steganographically encoded as a watermark in the media stream. ### Claim 5 (depends on 1) The method of claim 1, wherein receiving by the media client the identifier as in-band metadata with the media stream comprises receiving by the media client the identifier carried in a transport stream that carries the media stream to the media client. ### Claim 6 (depends on 1) The method of claim 1, wherein the out-of-band signaling comprises signaling over a broadband network connection. ### Claim 7 (depends on 1) The method of claim 1, wherein the media client comprises a content presentation device, and wherein the processing of the media stream for presentation comprises the media client presenting content of the media stream on a user interface. ### Claim 8 (depends on 1) The method of claim 1, wherein processing of the media stream for presentation comprises outputting the media stream for presentation by an associated device. ### Claim 9 (depends on 1) The method of claim 1, wherein enabling the server to obtain the reference fingerprints representing the media stream enables establishment of synchronous lock that maps reference time to client time, wherein establishing of the synchronous lock enables translating a reference time of the content-related action to a client time of the content-related action. ### Claim 10 (depends on 1) The method of claim 1, wherein enabling the server to obtain the reference fingerprints representing the media stream enables the server to perform fingerprint comparison between a replaceable content segment and the obtained reference fingerprints, to detect presence of the replaceable content segment in the media stream. ### Claim 11 (depends on 1) The method of claim 1, wherein the media stream comprises a television channel. ### Claim 12 A media client comprising: a content communication interface; a network communication interface; a processing unit; non-transitory data storage; and program instructions stored in the non-transitory data storage and executable by the processing unit to carry out operations including: receiving, through the content communication interface, a media stream, receiving, through the content communication interface, as in-band metadata with the media stream, an identifier of the media stream, processing the media stream for presentation, while processing the media stream for presentation, transmitting to a server, in out-of-band signaling through the network communication interface, the identifier of the media stream received as in-band metadata with the media stream, to enable the server to obtain reference fingerprints representing the media stream without a need for the server to search for reference fingerprints that match query fingerprints representing the media stream being processed for presentation, and receiving from the server, through the network communication interface, a directive for the media client to perform a content-related action at a particular time in the media stream, wherein the particular time is established based on the obtained reference fingerprints. ### Claim 13 (depends on 12) The method of claim 12, wherein the content-related action comprises dynamic ad insertion. ### Claim 14 (depends on 12) The media client of claim 12, wherein receiving the identifier as in-band metadata with the media stream comprises receiving the identifier steganographically encoded as a watermark in the media stream. ### Claim 15 (depends on 12) The media client of claim 12, wherein receiving by the media client the identifier as in-band metadata with the media stream comprises receiving by the media client the identifier carried in a transport stream that carries the media stream to the media client. ### Claim 16 (depends on 12) The media client of claim 12, wherein enabling the server to obtain the reference fingerprints representing the media stream enables establishment of synchronous lock that maps reference time to client time, wherein establishing of the synchronous lock enables translating a reference time of the content-related action to a client time of the content related action. ### Claim 17 (depends on 12) The media client of claim 12, wherein enabling the server to obtain the reference fingerprints representing the media stream enables the server to perform fingerprint comparison between a replaceable content segment and the obtained reference fingerprints, to detect presence of the replaceable content segment in the media stream. ### Claim 18 A method comprising: while a media client is processing a media stream for presentation, receiving by a server from the media client an out-of-band signal providing the server with an identifier of the media stream that the media client extracted from in-band metadata received by the media client with the media stream; using by the server the received identifier of the media stream as a basis to obtain reference fingerprints representing the media stream, without a need for the server to search for reference fingerprints that match query fingerprints representing the media stream being processed by the media client; and using by the server the obtained reference fingerprints as a basis to determine a point in the media stream when the media client should perform a content-related action; and directing by the server the media client to perform at the determined point the content-related action. ### Claim 19 (depends on 18) The method of claim 18, wherein the content-related action comprises dynamic ad insertion. ### Claim 20 (depends on 18) The method of claim 18, wherein the identifier of the media stream that the media client extracted from in-band metadata received by the media client with the media stream comprises an identifier that the media client extracted from a watermark in the media stream.