# US 2024/0205494 A1 · Gaze-responsive advertisement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0205494 A1
- Document ID: US20240205494A1
- Status: Published application
- Kind: A1
- Application number: 18/592,138
- Filed: 2024-02-29
- Published: 2024-06-20
- Earliest claimed: 2018-07-16
- Assignees: ARRIS Enterprises LLC
- Applicants: Arris Enterprises LLC
- Inventors: Anil Kumar NELLORE; Krishna Panje PANJE; Jeyakumar BARATHAN; Chandrashekar KSHEERASAGAR; Swaroop MAHADEVA
- Web page: https://patentgrep.com/patent/US20240205494A1

## CPC

- H04N 21/4318: {by altering the content in the rendering process, e.g. blanking, blurring or masking an image region (image enhancement or restoration in general G06T5/00)}
- G06Q 30/0269: {based on user profile or attribute}
- G06Q 30/0271: {Personalized advertisement}
- G06Q 30/0277: {Online advertisement}
- H04N 21/4223: Cameras (H04N23/00 takes precedence)
- H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)}
- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

This disclosure provides a system by which content/service providers can enable gaze-responsive advertisement/targeted content mechanisms for CPE devices like STB's/OTT streaming devices or systems or display devices. The system creates a new opportunity for content/service providers to advertise and helps in generating revenue. The system leverages using gaze-based technology in CPE devices like STB's or large display devices. A gaze-based algorithm helps the CPE devices to locate the viewer point of attention, using which gaze-responsive advertisement/content targeting can be achieved. The disclosure provides an end to end solution—from content creation to content consumption for gaze-responsive advertisement/media delivery.

## Claims

### Claim 1

An apparatus comprising:
an IP camera configured to continuously capture an image associated with a viewer; a receiver;
a gaze processor to identify view-gaze coordinates and gaze direction and determining a viewer attention area;
media content comprising a collection of gaze metadata;
a gaze metadata parser to parse the collection of gaze metadata; and
a gaze advertisement module to derive an area of interest (AOI) based on the collection of meta data,
wherein the AOI is compared to the viewer's attention area in order to select a gaze advertisement URL for display to the viewer.

### Claim 2

(canceled)

### Claim 3

(canceled)

### Claim 4

(canceled)

### Claim 5

(canceled)

### Claim 6

(canceled)

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the gaze metadata comprises (x,y) coordinates for each advertisable item, quadrant info, an advertisement URL for each advertisable item, a time frame for each advertisable item, and a video source resolution to which the (x,y) coordinates are relative.

### Claim 8

A system for creating gazed-based advertising content, the system comprising:
an image processing module comprising:
an incoming video stream;
an identification module to receive the incoming video stream and identify at least one advertisable item in a frame and provide a list of bitmaps for the at least one advertisable item;
an advertisement library comprising a list of advertisable items and a corresponding URL to each advertisable item, wherein the advertisable items are in a bitmap format; and
an ad mapper configured to compare the bitmaps from the identification module to the bitmaps from the advertisement library in order to provide a URL related to the at least one advertisable item,
wherein the image processing module outputs the advertisable item coordinates from the identification module and the URL associated with an advertisement associated with the at least one advertisable item.

### Claim 9

A method to insert advertisement data based on a user's gaze location, the method comprising:
receiving a video stream, the video stream including gaze metadata comprising (x,y) coordinates representing one or more advertisable items in the video stream;
detecting a gaze of a viewer;
using a gaze detection algorithm to determine (x,y) coordinates of the viewer attention on a display device;
calculating an Area of Interest AOI corresponding to each of the one or more advertisable items using the (x,y) coordinates of the gaze metadata and the video stream;
comparing the AOI with the (x,y) coordinates of the viewer attention;
determining if the AOI matches the viewer attention; and
if the AOI matches the viewer attention, selecting an advertisement URL; and
rendering an advertisement corresponding to the advertisement URL.

### Claim 10 (depends on 9)

The method of claim 9, wherein the method further comprises a pre-determined time threshold during the which the comparing the AOI with the (x,y) coordinates of the viewer attention is considered valid.

### Claim 11 (depends on 9)

The method of claim 9, wherein the gaze metadata comprises an advertisement URL for each advertisable item, a time frame for each advertisable item, and a video source resolution to which the (x,y) coordinates are relative.

### Claim 12

A method for gaze advertisement, the method comprising:
detecting a gaze associated with a viewer;
using a gaze detection algorithm to determine (x,y) coordinates of the viewer attention relative to a display device;
receiving a video stream, wherein the video stream comprises gaze metadata, the gaze metadata comprising frame quadrant location data corresponding to one or more advertisable items in the video stream;
determining a frame quadrant corresponding to to the (x,y) coordinates of the viewer's attention relative to the display;
selecting an advertisement URL based on the determined frame quadrant and the gaze metadata; and
rendering an advertisement corresponding to the advertisement URL.

### Claim 13

(canceled)

### Claim 14 (depends on 12)

The method of claim 12, wherein the method further comprises a pre-determined time threshold during the which the determining if the (x,y) coordinates of the viewer's attention to the display corresponds to the quadrant of the advertisable items is valid.

### Claim 15 (depends on 12)

The method of claim 12, wherein the gaze metadata comprises an advertisement URL for each advertisable item, a time frame for each advertisable item, and a video source resolution.

### Claim 16

A method for gaze advertisement insertion, the method comprising:
detecting a gaze direction of the at least one viewer relative to a display device, wherein the detection of the gaze is determined by head pose estimation methods;
receiving a video stream, wherein the video stream comprises gaze metadata, the gaze metadata comprising frame location data corresponding to one or more advertisable items in the video stream;
determining within the gaze metadata, if an advertisable item of the one or more advertised items is present in the left half of the frame or the right half of the frame;
determining if the viewer's gaze direction corresponds to the determined half of the frame which comprises the advertisable item of the one or more advertised items; and
rendering an advertisement if the gaze direction corresponds to the half of the frame which comprises the advertisable item of the one or more advertised items.

### Claim 17 (depends on 16)

The method of claim 16, wherein the method further comprises a pre-determined time threshold during the which determining if the viewer's gaze direction corresponds to the determined half of the frame which comprises the advertisable item of the one or more advertised items is valid.

### Claim 18 (depends on 16)

The method of claim 16, wherein the gaze metadata comprises an advertisement URL for each advertisable item, a time frame for each advertisable item, and a video source resolution.

### Claim 19 (depends on 9)

The method of claim 9, wherein the selecting the advertisement URL comprises:
transmitting the advertisement URL to an ad server; and
receiving, from the ad server, an advertisement corresponding to the advertisement URL.

### Claim 20 (depends on 12)

The method of claim 12, wherein the selecting the advertisement URL comprises:
transmitting the advertisement URL to an ad server; and
receiving, from the ad server, an advertisement corresponding to the advertisement URL.

### Claim 21 (depends on 16)

The method of claim 16, wherein the rendering the advertisement comprises:
transmitting the advertisement URL to an ad server; and
receiving, from the ad server, an advertisement corresponding to the advertisement URL.

### Claim 22

A system for to insert advertisement data based on a user's gaze location, the system comprising:
a processor configured to:
receive a video stream, the video stream including gaze metadata location data representing one or more advertisable items in the video stream;
detect a gaze of a viewer;
use a gaze detection algorithm to determine a gaze direction of the viewer;
calculate an Area of Interest AOI corresponding to each of the one or more advertisable items using the location data of the gaze metadata and the video stream;
compare the AOI with the gaze direction of the viewer;
determine if the AOI matches the gaze direction of the viewer; and
if the AOI matches the gaze direction of the viewer, select an advertisement URL; and
render an advertisement corresponding to the advertisement URL.

### Claim 23 (depends on 22)

The system of claim 22, wherein the location data comprises (x,y) coordinates for each advertisable item, an advertisement URL for each advertisable item, a time frame for each advertisable item, and a video source resolution to which the (x,y) coordinates are relative.

### Claim 24 (depends on 22)

The system of claim 22, wherein the location data comprises frame quadrant location data each advertisable item, an advertisement URL for each advertisable item, a time frame for each advertisable item, and a video source resolution.

### Claim 25 (depends on 23)

The system of claim 23, wherein the AOI corresponds to the (x,y) coordinates for each advertisable item, and the comparing the AOI with the gaze direction of the viewer comprises:
comparing the (x,y) coordinates for each advertisable item with the gaze direction of the viewer.

### Claim 26 (depends on 24)

The system of claim 24, wherein the AOI corresponds to the frame quadrant location for each advertisable item, and the comparing the AOI with the gaze direction of the viewer comprises:
comparing frame quadrant location for each advertisable item with the gaze direction of the viewer.
