# US 11,910,036 B2 · Systems and methods for targeted advertisement insertion into a program content stream > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,910,036 B2 - Document ID: US11910036B2 - Status: Granted - Kind: B2 - Application number: 18/097,116 - Filed: 2023-01-13 - Published: 2023-08-03 - Granted: 2024-02-20 - Earliest claimed: 2019-08-21 - Assignees: DISH Network L.L.C. - Applicants: Dish Network L.L.C. - Inventors: Robert W. Estus; Robert F. Percival - Web page: https://patentgrep.com/patent/US11910036B2 ## CPC - H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement} - H04N 21/458: Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules {; time-related management operations (arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10)} - H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)} - H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)} - H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A computerized system for targeted ad insertion receives a program content stream and detects images in the content stream. The system recognizes at least one visual object within the images and matches the recognized visual object to an advertisement that correlates with the recognized visual object. The system may then select an advertisement that advertises, correlates to, is directed to or is otherwise relevant to products or services falling under the theme, topic, category of interest and/or relevant viewer demographic and inserts an identifier of the matched advertisement at an insertion point into the program content stream. This enables playback of the matched advertisement during playback of the program content stream at a time that correlates to the insertion point into the program content stream of the identifier of the matched advertisement. ## Claims ### Claim 1 A computer implemented method for targeted ad insertion into a program content stream, comprising: receiving, by a computerized system for targeted ad insertion, the program content stream; detecting, by the computerized system for targeted ad insertion, images in the content stream; recognizing at least one visual object within the images; matching, by the computerized system for targeted ad insertion, the recognized at least one visual object within the images to an advertisement that correlates with the recognized at least one visual object in response to the matching the recognized at least one visual object within the images to an advertisement that correlates with the recognized at least one visual object, modifying an existing advertisement insertion cue in the program content stream by inserting, by the computerized system for targeted ad insertion, an identifier of the matched advertisement at an insertion point into the program content stream indicated by the existing advertisement insertion cue such that the modified advertisement insertion cue includes the identifier, the inserting of the identifier of the matched advertisement updating the program content stream enabling playback of the matched advertisement during playback of the program content stream at a time that correlates to the insertion point into the program content stream of the identifier of the matched advertisement; initiating, by the computerized system for targeted ad insertion, playback of the program content stream; recognizing in the program content stream, by the computerized system for targeted ad insertion, during playback of the program content stream, the identifier of the matched advertisement that was previously inserted at the insertion point into the program content stream; accessing, by the computerized system for targeted ad insertion, content of the matched advertisement by using the identifier of the matched advertisement that was previously inserted at the insertion point into the program content stream to search a database of advertisements during playback of the program content stream in response to the recognizing, during playback of the program content stream, the identifier of the matched advertisement that was previously inserted at the insertion point into the program content stream; and inserting, by the computerized system for targeted ad insertion, the content of the matched advertisement into the program content stream during the playback of the program content stream based on the identifier of the matched advertisement that was previously inserted at the insertion point into the program content stream. ### Claim 2 (depends on 1) The method of claim 1, further comprising: playing, by the computerized system for targeted ad insertion, the content of the matched advertisement during the playback of the program content stream at a time during playback of the program content stream that correlates to the insertion point into the program content stream of the identifier of the matched advertisement. ### Claim 3 (depends on 2) The method of claim 2, wherein the initiating playback of the program content stream; recognizing the identifier of the matched advertisement during playback of the program content stream; the accessing content of the matched advertisement in response to the recognizing the identifier of the matched advertisement during playback of the program content stream; and the inserting the content of the matched advertisement into the program content stream during the playback of the program content stream are performed by a receiving device at a customer premises. ### Claim 4 (depends on 2) The method of claim 2, wherein the playing the content of the matched advertisement during the playback of the program content stream at a time during playback of the program content stream that correlates to the insertion point into the program content stream of the identifier of the matched advertisement is performed by a receiving device at a customer premises. ### Claim 5 (depends on 1) The method of claim 1, wherein the receiving the program content stream; the detecting images in the content stream; the recognizing at least one visual object within the images; the matching the recognized at least one visual object within the images to an advertisement that correlates with the recognized at least one visual object; and the inserting, the identifier of the matched advertisement at an insertion point into the program content stream, enabling playback of the matched advertisement during playback of the program content stream at a time that correlates to the insertion point into the program content stream of the identifier of the matched advertisement are performed by a content server outside a customer premises. ### Claim 6 (depends on 1) The method of claim 1, wherein the recognizing the at least one visual object within the images includes: detecting a plurality of visual object candidates within the images; determining at least one of the visual object candidates is a type of visual object associated with one or more of a plurality of advertisements; and recognizing the at least one of the visual object candidates as the at least one visual object within the images based on the determination. ### Claim 7 (depends on 1) The method of claim 1, wherein the matching the recognized at least one visual object within the images to an advertisement that correlates with the recognized at least one visual object includes: accessing a plurality of stored advertisement data corresponding to a plurality of different advertisements; determining which advertisement of the plurality of advertisements advertises a product or service that correlates with the recognized at least one visual object within the images based on the accessing of the plurality of stored advertisement data; and matching the recognized at least one visual object within the images to the determined advertisement of the plurality of advertisements. ### Claim 8 (depends on 1) The method of claim 1, further comprising: detecting, by the computerized system for targeted ad insertion, audio in the content stream; recognizing, by the computerized system for targeted ad insertion, at least one characteristic of the audio; matching, by the computerized system for targeted ad insertion, the recognized at least one characteristic of the audio to an advertisement that correlates with the recognized at least one characteristic of the audio; and inserting, by the computerized system for targeted ad insertion, an identifier of the matched advertisement at an insertion point into the program content stream, enabling playback of the matched advertisement during playback of the program content stream at a time that correlates to the insertion point into the program content stream of the identifier of the matched advertisement. ### Claim 9 A system, comprising: at least one computer processor; and at least one memory coupled to the at least one computer processor, the at least one memory having computer-executable instructions stored thereon that, when executed by the at least one computer processor, cause the at least one computer processor to: initiate playback of a program content stream; recognize in the program content stream, an identifier of a matched advertisement during playback of the program content stream, the identifier of the matched advertisement having been inserted at an insertion point into the program content stream in response to at least one visual object recognized within images of the content stream being matched to the advertisement which correlates with the recognized at least one visual object access content of the matched advertisement by using the identifier of the matched advertisement that was previously inserted at the insertion point into the program content stream to search a database of advertisements during playback of the program content stream in response to the recognizing, during playback of the program content stream, the identifier of the matched advertisement that was previously inserted at the insertion point into the program content stream; and insert the content of the matched advertisement into the program content stream during the playback of the program content stream based on the identifier of the matched advertisement that was previously inserted at the insertion point into the program content stream. ### Claim 10 (depends on 9) The system of claim 9, wherein the computer-executable instructions, when executed by the at least one computer processor, further cause the at least one computer processor to: play the content of the advertisement during the playback of the program content stream at a time during playback of the program content stream that correlates to the insertion point into the program content stream of the identifier of the advertisement. ### Claim 11 (depends on 10) The system of claim 10, wherein the accessing content of the matched advertisement includes accessing content of the matched advertisement at a set-top box from a remote advertisement server. ### Claim 12 (depends on 10) The system of claim 10, wherein the accessing content of the matched advertisement includes accessing content of the matched advertisement by a set-top box from local storage of the set-top box. ### Claim 13 (depends on 9) The system of claim 9, wherein the computer-executable instructions, when executed by the at least one computer processor, further cause the at least one computer processor to: receive a plurality of stored advertisement data corresponding to a plurality of different advertisements; search the plurality of stored advertisement data corresponding to a plurality of different advertisements for advertisement data that is identified by the identifier of the matched advertisement during playback of the program content stream; and access content of the matched advertisement in response to the recognizing the identifier of the advertisement during playback of the program content stream based on the advertisement data that is identified by the identifier of the matched advertisement during playback of the program content stream. ### Claim 14 A non-transitory computer-readable storage medium having computer-executable instructions stored thereon that, when executed by at least one processor, cause the at least one processor to: access a plurality of stored advertisement data corresponding to a plurality of different advertisements; determine which advertisement of the plurality of advertisements advertises a product or service that correlates with at least one visual object recognized within images of a program content stream based on the accessing of the plurality of stored advertisement data; match the at least one visual object recognized within the images to the determined advertisement of the plurality of advertisements; insert an identifier of the determined advertisement of the plurality of advertisements at an insertion point into the program content stream, enabling playback of the determined advertisement of the plurality of advertisements during playback of the program content stream at a time that correlates to the insertion point into the program content stream of the identifier of the determined advertisement of the plurality of advertisements; initiating playback of the program content stream; recognizing in the program content stream during playback of the program content stream, the identifier of the determined advertisement that was previously inserted at the insertion point into the program content stream; accessing content of the determined advertisement by using the identifier of the determined advertisement that was previously inserted at the insertion point into the program content stream to search a database of advertisements during playback of the program content stream in response to the recognizing, during playback of the program content stream, the identifier of the determined advertisement that was previously inserted at the insertion point into the program content stream; and inserting, by a computerized system for targeted ad insertion, the content of the determined advertisement into the program content stream during the playback of the program content stream based on the identifier of the determined advertisement that was previously inserted at the insertion point into the program content stream.