Back to results
US 11,308,321 B2Granted
Method and system for 3D cornea position estimation
AssigneeTobii AB
InventorsDavid Masko, Magnus Ivarsson, Niklas Ollesson, Anna Redz
Life of this patent
Estimating the term…
Abstract
There is provided a method, system, and non-transitory computer-readable storage medium for performing three-dimensional, 3D, position estimation for the cornea center of an eye of a user, using a remote eye tracking system, wherein the position estimation is reliable and robust also when the cornea center moves over time in relation to an imaging device associated with the eye tracking system. This is accomplished by generating, using, and optionally also updating, a cornea movement filter, CMF, in the cornea center position estimation.
Claims16
A method for performing three-dimensional, 3D, position estimation for a cornea center of an eye of a user, using a remote eye tracking system, when the cornea center moves over time in relation to an imaging device associated with the eye tracking system, the method comprising:
- generating, using processing circuitry associated with the eye tracking system, a cornea movement filter, CMF, comprising an estimated initial 3D position and an estimated initial 3D velocity of the cornea center of the eye at a first time instance, wherein the cornea movement filter, CMF, is a Kalman filter;
- predicting, using the processing circuitry:
- a first two-dimensional, 2D, glint position in an image captured at a second time instance by applying the cornea movement filter, CMF, wherein the predicted first glint position represents a position where a first glint is predicted to be generated by a first illuminator associated with the eye tracking system;
- and
- a second 2D glint position in the image captured at the second time instance by applying the cornea movement filter, CMF, wherein the predicted second glint position represents a position where a glint is predicted to be generated by a second illuminator associated with the eye tracking system;
- performing image processing, using the processing circuitry, including the steps of:
- identifying at least one first candidate glint in a first image captured by the imaging device at the second time instance wherein the first image comprises at least part of the cornea of the eye and at least one glint generated by the first illuminator;
- and
- identifying at least one second candidate glint in the first image or in a second image captured by the imaging device at a third time instance and comprising at least part of the cornea of the eye and at least one glint generated by the second illuminator;
- selecting, using the processing circuitry, a pair of a first and second candidate glint, out of all possible pairs of a first candidate glint and a second candidate glint, that has a highest probability of corresponding to the predicted first and second glint positions, based on a probability estimation function;
- estimating a current 3D position of the cornea, using the processing circuitry, based on positions of the selected pair of first and second candidate glints;
- and updating the cornea movement filter, CMF, using the processing circuitry, by setting the estimated initial 3D position, of the cornea center to the current 3D position of the cornea center;
- and
- determining a current estimated 3D velocity of the cornea center based on the estimated initial 3D position of the cornea center and the current 3D position of the cornea center;
- and
- setting the estimated initial 3D velocity of the cornea center to the current estimated 3D velocity of the cornea center.
13 more elements
An eye tracking system for performing three-dimensional, 3D, position estimation for a cornea center of an eye of a user when the cornea center moves over time in relation to an imaging device associated with the eye tracking system, the eye tracking system comprising processing circuitry configured to:
- generate a cornea movement filter, CMF, comprising an estimated initial 3D position and an estimated initial 3D velocity of the cornea center of the eye at a first time instance, wherein the cornea movement filter, CMF, is a Kalman filter;
- predict a first two-dimensional, 2D, glint position in an image captured at a second time instance by applying the cornea movement filter, CMF, wherein the predicted first glint position represents a position where a first glint is predicted to be generated by a first illuminator associated with the eye tracking system;
- and
- predict a second 2D glint position in the image captured at the second time instance by applying the cornea movement filter, CMF, wherein the predicted second glint position represents a position where a glint is predicted to be generated by a second illuminator associated with the eye tracking system;
- perform image processing to:
- identify at least one first candidate glint in a first image captured by the imaging device at the second time instance, wherein the first image comprises at least part of the cornea of the eye and at least one glint generated by the first illuminator;
- and
- identify at least one second candidate glint in the first image or in a second image captured by the imaging device at a third time instance and comprising at least part of the cornea of the eye and at least one glint generated by the second illuminator;
- select a pair of a first and second candidate glint, out of all possible pairs of a first candidate glint and a second candidate glint, that has a highest probability of corresponding to the predicted first and second glint positions, based on a probability estimation function;
- estimate a current 3D position of the cornea based on positions of the selected pair of first and second candidate glints;
- and
- update the cornea movement filter, CMF, by setting the estimated initial 3D position of the cornea center to the current 3D position of the cornea center;
- and
- determine a current estimated 3D velocity of the cornea center based on the estimated initial 3D position of the cornea center and the current 3D position of the cornea center;
- and
- set the estimated initial 3D velocity of the cornea center to the current estimated 3D velocity of the cornea center.
13 more elements
A non-transitory computer-readable storage medium storing instructions which, when executed by processing circuitry of an eye tracking system, cause the eye tracking system to:
- generate a cornea movement filter, CMF, comprising an estimated initial 3D position and an estimated initial 3D velocity of a cornea center of an eye at a first time instance, wherein the cornea movement filter, CMF, is a Kalman filter;
- and
- predict a first two-dimensional, 2D, glint position in an image captured at a second time instance by applying the cornea movement filter, CMF, wherein the predicted first glint position represents a position where a first glint is predicted to be generated by a first illuminator associated with the eye tracking system;
- predict a second 2D glint position in the image captured at the second time instance by applying the cornea movement filter, CMF, wherein the predicted second glint position represents a position where a glint is predicted to be generated by a second illuminator associated with the eye tracking system;
- perform image processing to:
- identify at least one first candidate glint in a first image captured by an imaging device at the second time instance wherein the first image comprises at least part of the cornea of the eye and at least one glint generated by the first illuminator;
- and
- identify at least one second candidate glint in the first image or in a second image captured by the imaging device at a third time instance and comprising at least part of the cornea of the eye and at least one glint generated by the second illuminator;
- select a pair of a first and second candidate glint, out of all possible pairs of a first candidate glint and a second candidate glint that has a highest probability of corresponding to the predicted first and second glint positions, based on a probability estimation function;
- estimate a current 3D position of the cornea based on positions of the selected pair of first and second candidate glints;
- and
- update the cornea movement filter, CMF, by setting the estimated initial 3D position of the cornea center to the current 3D position of the cornea center;
- determine a current estimated 3D velocity of the cornea center based on the estimated initial 3D position of the cornea center and the current 3D position of the cornea center;
- and
- set the estimated initial 3D velocity of the cornea center to the current estimated 3D velocity of the cornea center.
12 more elements
Dependencies are inferred from the claim text.