# US 11,910,057 B2 · Environment object recognition > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,910,057 B2 - Document ID: US11910057B2 - Status: Granted - Kind: B2 - Application number: 17/077,589 - Filed: 2020-10-22 - Published: 2021-06-10 - Granted: 2024-02-20 - Earliest claimed: 2013-02-25 - Assignees: Comcast Cable Communications, LLC - Applicants: Comcast Cable Communications, LLC - Inventors: Dan E Holden; Michael J. Cook - Web page: https://patentgrep.com/patent/US11910057B2 ## CPC - H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)} - H04N 21/4223: Cameras (H04N23/00 takes precedence) - H04N 21/458: Scheduling content for creating a personalised stream, e.g. by combining a locally stored advertisement with an incoming stream; Updating operations, e.g. for OS modules {; time-related management operations (arrangements for replacing or switching information during the broadcast or during the distribution H04H20/10)} - H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Methods, systems, and devices for displaying additional or targeted content to a user are disclosed. An image or data capture device associated with a display device may capture an image of a space associated with the user or capture data related to what is being viewed by the viewer's eyes. Objects in the captured image or objects being viewed may be recognized, including any identifiable image, item, detail, or article and the like. A recognized object may then be compared to a target list to determine a match. If matched, and if there is additional content associated with the target object, the additional content may be presented to the user. ## Claims ### Claim 1 A method comprising: causing output, via a display in an environment, of video content; receiving two different images of the environment, wherein the two different images correspond to two different times the display was outputting the video content; selecting additional video content based on a target list associating a first image object with the additional video content and a second image object, and based on determining, via image processing of the two different images, that: the first image object is present in an earlier image of the two different images; and the first image object is absent, but the second image object is present, in a later image of the two different images; and causing output of the additional video content via the display. ### Claim 2 (depends on 1) The method of claim 1, wherein the selecting comprises: executing eye tracking to determine a user's eye focus in each of the two different images; determining that the user's eye focus has changed between the two different images; accessing information indicating different selectable video content corresponding to different display portions; and selecting the additional video content based on a change in the user's eye focus and based on the information. ### Claim 3 (depends on 1) The method of claim 1, wherein the selecting comprises executing eye tracking to determine a user's eye focus in each of the two different images, and wherein the selecting the additional video content is further based on the user's eye focus. ### Claim 4 (depends on 1) The method of claim 1, wherein the video content comprises a plurality of video programs. ### Claim 5 (depends on 1) The method of claim 1, further comprising: tracking eye focus of a user during output of the video content via the display; and causing output of different additional video content based on the eye focus targeting different content corresponding to different image regions of the video content. ### Claim 6 (depends on 1) The method of claim 1, wherein the target list indicates that a message, from a first product brand, is to be output if a product, from a second product brand, replaces a product of the first product brand in the environment. ### Claim 7 (depends on 1) The method of claim 1, wherein the first image object comprises a product having a first name, the second image object comprise an object having a second name, and wherein the additional video content comprises a message, identifying the first name, that is to be output based on determining that the first name no longer appears in the environment. ### Claim 8 A method comprising: using a camera to capture a plurality of images during output of media content, wherein the plurality of images comprise images of a user and an environment in which the media content is being output; causing computerized image processing of the plurality of images to determine that an object has been removed from the environment; selecting, based on the determining that the object has been removed from the environment and based on a target list associating the object with additional content, the additional content to be output for the user; and causing output of the selected additional content. ### Claim 9 (depends on 8) The method of claim 8, further comprising: executing eye tracking to determine eye focus of the user in the plurality of images; determining that the eye focus has changed, in the plurality of images, between two portions of a display via which the media content was output; and accessing information indicating different selectable items corresponding to different portions of the display, wherein the selecting the additional content is based on the information and the change in eye focus. ### Claim 10 (depends on 8) The method of claim 8, further comprising executing eye tracking to determine eye focus of the user in the plurality of images, wherein the selecting the additional content is further based on the eye focus. ### Claim 11 (depends on 8) The method of claim 8, wherein the media content comprises a plurality of video programs. ### Claim 12 (depends on 8) The method of claim 8, further comprising: tracking eye focus of the user during output of the media content via a display; and causing output of different additional content based on the eye focus targeting different objects in the media content. ### Claim 13 (depends on 8) The method of claim 8, wherein the selecting the additional content is further based on determining that an eye focus of the user has remained constant for a minimum duration of time. ### Claim 14 (depends on 8) The method of claim 8, wherein the target list indicates: a first identifier visible on the object; and that the associated additional content is to be output if the first identifier is replaced with a second object, having a second identifier, in the environment. ### Claim 15 A method comprising: causing output, via a display in an environment, of a video program; executing eye tracking, of eyes of a user viewing the video program, to determine user eye focus; accessing information indicating: different content corresponding to different image regions of the video program; and that an object was previously in the environment, but is no longer in the environment; selecting, by a computing device and based on the user eye focus and the information, an additional video program; and causing output of the additional video program via the display. ### Claim 16 (depends on 15) The method of claim 15, wherein the information also indicates content corresponding to objects recognized in the environment outside of the display. ### Claim 17 (depends on 15) The method of claim 15, wherein the eye tracking comprises tracking different quantities of time that the user focuses on different objects in the video program. ### Claim 18 (depends on 15) The method of claim 15, wherein the selecting is also based on determining that the user eye focus has remained on a region of the display for a minimum quantity of time. ### Claim 19 (depends on 15) The method of claim 15, wherein the selecting is also based on image recognition of an object in an image of the environment. ### Claim 20 (depends on 15) The method of claim 15, wherein the selecting is also based on a target list identifying the object and the additional video program, and wherein the target list indicates that the additional video program is to be output based on the object being replaced, in the environment, by a second object. ### Claim 21 (depends on 15) The method of claim 15, wherein the additional video program is a video advertisement different from the video program. ### Claim 22 (depends on 15) The method of claim 15, wherein the additional video program comprises audio and video that are different from the video program.