Real time eye tracking for human computer interaction
AssigneeThe Research Foundation for The State University of New York
InventorsLijun Yin, Michael Reale
Life of this patent
Estimating the term…
Abstract
A gaze direction determining system and method is provided. A two-camera system may detect the face from a fixed, wide-angle camera, estimates a rough location for the eye region using an eye detector based on topographic features, and directs another active pan-tilt-zoom camera to focus in on this eye region. A eye gaze estimation approach employs point-of-regard (PoG) tracking on a large viewing screen. To allow for greater head pose freedom, a calibration approach is provided to find the 3D eyeball location, eyeball radius, and fovea position. Both the iris center and iris contour points are mapped to the eyeball sphere (creating a 3D iris disk) to get the optical axis; then the fovea rotated accordingly and the final, visual axis gaze direction computed.
Claims20
A method for estimating an eyeball gaze direction based on an image of a face, comprising:
- determining, from the image, a center of an iris of the eyeball by:
- estimating an outline of the iris in the image,
- mapping the outline of the iris as a set of perspective vectors intersecting a center of a spherical model of the eyeball,
- defining a set of iris contour vectors representing normalized vectors extending from the center of the eyeball spherical model to intersecting points of the perspective vectors,
- computing a statistical average vector of the iris contour vectors as the estimated optic axis of the eyeball, wherein a location of the fovea of the eyeball lies displaced from the optic axis of the eyeball;
- estimating, from the image, the eyeball gaze direction along a vector, displaced from the optic axis of the eyeball, from a predicted location of the fovea and passing through the determined center of the iris, based on at least:
- the determined center of the iris,
- the center of the spherical model of the eyeball, and
- the radius of the spherical model of the eyeball,
- by rotating an estimated location of the fovea offset based on the optic axis of the eyeball;
- and
- calibrating the determined center of the iris and resulting estimated eyeball gaze direction with respect to a known gaze direction, to update parameters defining the estimated eyeball gaze direction for unknown gaze directions.
An apparatus for eyeball gaze direction estimation, comprising:
- at least one camera configured to generate an image of a face;
- and
- at least one automated processor, configured to:
- determine, from the image, a center of an iris of the eyeball, based on at least:
- an estimate of an outline of the iris in the image,
- the outline of the iris mapped as a set of perspective vectors intersecting a center of a spherical model of the eyeball, and
- a set of iris contour vectors representing normalized vectors extending from the center of the eyeball spherical model to intersecting points of the perspective vectors, and
- compute a statistical average vector of the iris contour vectors as the estimated optic axis of the eyeball,
- wherein a location of the fovea of the eyeball lies displaced from the optic axis of the eyeball;
- estimate, from the image, the eyeball gaze direction along a vector, displaced from the optic axis of the eyeball, from a predicted location of the fovea and passing through the determined center of the iris, based on at least:
- the determined center of the iris,
- the center of the spherical model of the eyeball, and
- the radius of the spherical model of the eyeball,
- by a rotation of an estimated location of the fovea offset based on the optic axis of the eyeball;
- and
- calibrate the determined center of the iris and resulting estimated eyeball gaze direction with respect to a known gaze direction, to update parameters defining the estimated eyeball gaze direction for unknown gaze directions.
A method for eyeball gaze direction vector estimation, comprising:
- acquiring images of a face;
- determining an iris contour and a center of the iris within the images based on an outline of the iris in the images, the outline of the iris being mapped as a set of perspective vectors intersecting a center of the eyeball, a set of iris contour vectors being defined representing normalized vectors extending from the center of the eyeball to intersecting points of the iris contour, such that the center of the iris is determined as a statistical average of the normalized vectors;
- estimating a three dimensional center and radius of the eyeball, wherein a fovea of the eyeball is displaced from the optic axis of the eyeball defined by a vector extending from the three dimensional center of the iris through the three dimensional center of the eyeball;
- estimating an eyeball gaze direction along a vector from a predicted position of the fovea and passing through the center of the iris, based on at least the determined center of the iris and the center and the radius of the eyeball;
- and
- updating a set of stored parameters for estimating at least the three dimensional center and the radius of the eyeball and a position of the fovea, based on acquired images of the eyeball when gazing toward at least two distinct predetermined objects.
Dependencies are inferred from the claim text.