# US 2018/0024632 A1 · Interactive Display System with Eye Tracking to Display Content According to Subject's Interest

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0024632 A1
- Document ID: US20180024632A1
- Status: Published application
- Kind: A1
- Application number: 15/657,064
- Filed: 2017-07-21
- Published: 2018-01-25
- Earliest claimed: 2016-07-21
- Applicants: Aivia, Inc.
- Inventors: Chungwen Dennis Lo
- Web page: https://patentgrep.com/patent/US20180024632A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- A47F 13/00: Shop or like accessories (devices for drawing-off and cutting paper from a roll B65H; movably-mounted ladders E06C9/06)
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/1423: {controlling a plurality of local displays, e.g. CRT and flat panel display}
- G06Q 30/0241: Advertisements
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06V 40/166: {using acquisition arrangements}
- G06V 40/19: Sensors therefor
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- H04N 7/183: {for receiving images from a single remote source}
- H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- A47F 2010/025: {using stock management systems}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 2203/04803: Split screen, i.e. subdividing the display area or the window area into separate subareas
- G06T 7/90: Determination of colour characteristics
- G06T 2207/10024: Color image
- G06T 2207/30201: Face
- G06T 2207/30232: Surveillance
- G09G 5/32: with means for controlling the display position {(see provisionally G09G5/42)}
- G09G 2320/0693: Calibration of display systems
- G09G 2354/00: Aspects of interface with display user
- G09G 2370/022: Centralised management of display operation, e.g. in a server instead of locally
- G09G 2380/06: Remotely controlled electronic signs other than labels
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A47F: SPECIAL FURNITURE, FITTINGS, OR ACCESSORIES FOR SHOPS, STOREHOUSES, BARS, RESTAURANTS OR THE LIKE; PAYING COUNTERS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A system interactively displays content according to subject's interest. An interactive display system includes a display and an imaging unit or camera. The interactive display system tracks a subject's eyes or head movement to determine a subject's interest. Then, the system will analyze the subject's behavior and make decisions on what content to display on a screen based on the subject's interest.

## Claims

### Claim 1

A kit comprising:
at least a first imaging device; and
a controller device, wherein the controller device comprises a display adapter configured to be coupled to a first display and a port configured to be coupled to the imaging device, code executable on a processor of the controller device, and controller device comprises:
code to acquire images from the imaging device;
code to analyze the images from the imaging device to obtain a first analysis; and
code alter the content shown on the first display based on the first analysis of the images, wherein the content shown on the first display does not comprise images acquired from the imaging device.

### Claim 2

The kit wherein the controller device comprises:
a network adapter, configure to be coupled to a communications network, wherein the controller transmits the first analysis to a server through the network adapter and the communications network; and
the controller block is configured to cause a second display, coupled to the network and separate from the first display, to show a content based on the first analysis.

### Claim 3 (depends on 1)

The kit of claim 1 wherein the analyze the images from the imaging device to obtain a first analysis comprises the controller block being configured to:
detect a gaze event of a person, wherein the gaze event indicates a selection by the person's eye gaze of either at least a first content or a second content shown on the first display;
upon determining the gaze event is for the first content, display a third content associated with the first content on the first display; and
upon determining the gaze event is for the second content, display a fourth content associated with the second content on the first display.

### Claim 4 (depends on 1)

The kit of claim 1 wherein the controller is configured to calibrate based on a point of interest at about a frame center, between a frame left edge and a frame right edge of the first display and between a frame top edge and a frame bottom edge of the first display.

### Claim 5 (depends on 1)

The kit of claim 1 wherein the controller is configured to calibrate based on a point of interest moving from a frame left edge to a frame right edge of the first display or the point of interest moving from the frame right edge to the frame left edge of the first display.

### Claim 6 (depends on 1)

The kit of claim 1 wherein the controller is configured to calibrate based on a point of interest at one side of a frame of the first display and then at an opposite side of the frame of the first display.

### Claim 7 (depends on 1)

The kit of claim 1 wherein the controller comprises a real-time processor, and the processor is configured to perform image analysis of gaze click detection, group classification, movement detection, and location estimation.

### Claim 8 (depends on 1)

The kit of claim 1 wherein the controller comprises embedded storage or is coupled to external storage, the storage is used to store content images received from a server for a presenter and a reporter, and the controller based on image analysis data, determines associated display content.

### Claim 9 (depends on 1)

The kit of claim 1 wherein the image analysis comprises the controller being configured to determine a gaze duration, estimate face location, and generate a gaze_click flag when a duration is greater than a predetermined threshold time value.

### Claim 10 (depends on 1)

The kit of claim 1 wherein the image analysis comprises the controller being configured to detect a movement of a person's eyes, a movement of a person's head, and movement of a person's body, a person's gender, a person's age, a person's movement behavior or patterns, a person's distance from first display, a person's hair color, a person's clothing color, a person's clothing style such as pants, skirt, or other, appearance, posture, face recognition or face tracking, or any combination of these.

### Claim 11 (depends on 1)

The kit of claim 1 wherein the alter the content shown on the first display is enabled by generating of a gaze_click through flag which comprises a gaze_click and weighting factors comprising at least one of a specific gender, age group, specific area, distance, preferred viewers, or other factors, or any combination of these.

### Claim 12 (depends on 1)

The kit of claim 1 wherein the altered content is migrated from a primary content group to secondary content group to match to a classified viewer's group.

### Claim 13 (depends on 1)

The kit of claim 1 wherein the altered content can be a content size updated according to viewer's distance, content color to match viewer's dress color, content waving to get attention, content still from waving, or different content, or any combination of these.

### Claim 14 (depends on 1)

The kit of claim 1 wherein the imaging device located in a separate location than the first display.

### Claim 15 (depends on 1)

The kit of claim 1 wherein the imaging device is positioned in at least one of a mannequin, a merchandise, a holder, or a stand, separate from the first display.

### Claim 16 (depends on 1)

The kit of claim 1 wherein the imaging device incorporates a motor to rotate the imaging device itself or to rotate a front mirror to increase its field of view.

### Claim 17 (depends on 1)

The kit of claim 1 wherein multiple display units and imaging units are linked together, such that when a subject moves from a display unit A's coverage to a display unit B's coverage, a subject's eye is tracked such that display unit B will display content that related to what was displayed in unit A when the subject's gaze was detected in unit A.

### Claim 18 (depends on 1)

The kit of claim 1 wherein for each instance content is displayed on the first display, captured images associated with the content on the first display are analyzed to determine an interest level, where lower interest content will be replaced with content similar to high interest content either in a single display or using multiple display units.

### Claim 19 (depends on 1)

The kit of claim 1 wherein the image analysis comprises a gaze blinking which is used to determine if a real human.