# US 2016/0249800 A1 · Computational array camera with dynamic illumination for eye tracking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0249800 A1
- Document ID: US20160249800A1
- Status: Published application
- Kind: A1
- Application number: 15/150,949
- Filed: 2016-05-10
- Published: 2016-09-01
- Earliest claimed: 2014-03-28
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Nizan Horesh; Amir Bousany
- Web page: https://patentgrep.com/patent/US20160249800A1

## CPC

- G06V 40/18: Eye characteristics, e.g. of the iris
- A61B 3/0025: {characterised by electronic signal processing, e.g. eye models}
- A61B 3/107: for determining the shape or measuring the curvature of the cornea
- A61B 3/113: for determining or recording eye movement
- A61B 3/14: Arrangements specially adapted for eye photography
- G06V 40/193: {Preprocessing; Feature extraction}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

Methods and systems for eye tracking are disclosed. One such method obtains a plurality of images of the eye from an array camera and detects glint positions and a pupil edge of the eye in the plurality of the images. A distance from the array camera to the pupil edge may be estimated based on the glint positions. A pupil image may be generated based on selected ones of the plurality of images, the glint positions, and the estimated distance to the pupil edge. A pupil center position may be determined based on the pupil image and the glint positions.

## Claims

### Claim 1

(canceled)

### Claim 2

A method for eye tracking comprising:
obtaining a plurality of images comprising glints as generated by an illumination pattern;
detecting a pupil edge in the plurality of images;
determining a subset of light sources that reduces interference of the glints with the pupil edge while keeping a plurality of glints;
estimating a center of the pupil; and
tracking a gaze vector based on the estimate of the center of the pupil.

### Claim 3 (depends on 2)

The method of claim 2, further comprising detecting the eye by:
determining a region-of-interest, including the eye, for searching;
adjusting the illumination pattern based on the region-of-interest;
detecting the illumination pattern in the plurality of images as represented by a glint pattern; and
tracking the eye based on the glint pattern.

### Claim 4 (depends on 3)

The method of claim 3, wherein adjusting the illumination pattern comprises activating and/or deactivating particular light sources such that only light sources producing glints in the region of interest are activated.

### Claim 5 (depends on 2)

The method of claim 2, wherein determining the subset of light sources that reduces interference of the glints with the pupil edge comprises deactivating particular light sources that interfere with the pupil edge.

### Claim 6 (depends on 2)

The method of claim 2, wherein determining the subset of light sources that reduces interference of the glints with the pupil edge comprises estimating a variance of the estimate of the center of the pupil by an inverse of a Fisher information matrix of ellipse parameters.

### Claim 7 (depends on 2)

The method of claim 2, further comprising estimating a distance from an array camera to the eye based on the glints and triangulation.

### Claim 8 (depends on 2)

The method of claim 2, further comprising generating the illumination patter from a plurality of infrared light sources.

### Claim 9 (depends on 2)

The method of claim 2, further comprising generating a pupil image by selecting one of the plurality of images in which the pupil edge is not obscured by a glint.

### Claim 10

A non-transitory machine-readable medium comprising instructions for eye tracking, which when executed by a machine, cause the machine to:
obtain a plurality of images comprising glints as generated by an illumination pattern;
detect a pupil edge in the plurality of images;
determine a plurality of light sources that reduces interference of the glints with the pupil edge while keeping a plurality of glints;
estimate a center of the pupil; and
track a gaze vector based on the estimate of the center of the pupil.

### Claim 11 (depends on 10)

The machine readable medium of claim 10, wherein the machine is further caused to estimate a distance of the pupil edge from an array camera based on glint positions in each image, a location of the array camera relative to light sources, corneal curvature of the eye, and a three-dimensional position of a corneal center of curvature.

### Claim 12 (depends on 10)

The machine readable medium of claim 10, wherein the machine is further caused to determine the gaze vector based on the pupil center position, the glint positions, and one of an eyeball model using Purkinje images, a global head model, or an active appearance model.

### Claim 13 (depends on 10)

The machine readable medium of claim 10, wherein the machine is further caused to activate or deactivate particular ones of the plurality of light sources based on detected interference of one or more glints, resulting from respective ones of the plurality of light sources, with the pupil edge.

### Claim 14 (depends on 13)

The machine readable medium of claim 13, wherein the plurality of light sources are infrared light sources.

### Claim 15 (depends on 10)

The machine readable medium of claim 10, wherein the machine is further caused to adjust the illumination pattern by activating and/or deactivating particular one of the plurality of light sources such that only light sources producing glints in the region of interest are activated.

### Claim 16 (depends on 15)

The machine readable medium of claim 15, wherein determining the subset of light sources that reduces interference of the glints with the pupil edge comprises deactivating particular light sources of the plurality of light sources that interfere with the pupil edge.

### Claim 17

An eye tracking system comprising:
an array camera to generate a plurality of images of the eye;
a plurality of light sources to generate an illumination pattern represented by a glint pattern on the eye; and
a processing unit, coupled to the array camera and the plurality of light sources, to obtain the plurality of images, detect a pupil edge in the plurality of images, determine a subset of light sources of the plurality of light sources that reduces interference of the glints with the pupil edge while keeping a plurality of glints, estimate a center of the pupil, and track a gaze vector based on the estimate of the center of the pupil.

### Claim 18 (depends on 17)

The eye tracking system of claim 17, wherein the processing unit is further to control deactivation of particular ones of the subset of light sources in response to a glint of the glint pattern interfering with the pupil edge in at least one of the plurality of images.

### Claim 19 (depends on 17)

The eye tracking system of claim 17, wherein the processing unit is further to determine a region-of-interest within a field of view of the array camera and adjust the illumination pattern by activating or deactivating particular ones of the plurality of light sources to keep the illumination pattern within the region of interest.

### Claim 20 (depends on 17)

The eye tracking system of claim 17, wherein the processing unit is further to generate the pupil image from the plurality of images such that the pupil image has a higher resolution than any one of the plurality of images.

### Claim 21 (depends on 17)

The eye tracking system of claim 17, wherein the processing unit is further to generate a three-dimensional position of the pupil edge based on triangulation from parallax error of the pupil edge in at least a pair of images of the plurality of images.
