Eye tracking using a light field camera on a head-mounted display

A head mounted display (HMD) comprising:

• a display element configured to display content to a user wearing the HMD;
• a light source configured to illuminate an eye, inside the HMD, of the user;
• a light field camera configured to capture a plenoptic image of the eye;
• and
• a controller configured to:
• update a light field model of the eye based on the plenoptic image of the eye;
• identify a region of a plane that includes an iris of the eye in the light field model by:
• determining the plane by approximating a curvature of a surface of the eye based on depth information from the plenoptic image;
• and
• identifying the region of the plane based on a size of the iris of the eye;
• and
• determine a gaze direction of the eye based on the identified region.

A head mounted display (HMD) comprising:

• a display element configured to display content to a user wearing the HMD;
• an optics block configured to direct light from the display element to an exit pupil of the HMD;
• a light field camera configured to capture a plenoptic image of an eye of the user;
• and
• a controller configured to:
• receive the plenoptic image of the eye from the light field camera;
• provide the plenoptic image of the eye to a virtual reality (VR) console;
• receive, from the VR console, a light field model of the eye generated based on the plenoptic image of the eye, the VR console configured to:
• determine a plane of the eye in the light field model by approximating a curvature of a surface of the eye based on depth information from the plenoptic image;
• and
• identify a region of the plane, including an iris of the eye in the light field model, based on a size of the iris of the eye;
• and
• determine a gaze direction of the eye based on the identified region of the light field model.

An eye tracking system comprising:

• a light field camera configured to capture a plenoptic image of an eye of a user;
• and
• a controller configured to:
• update a light field model of the eye based on the plenoptic image of the eye;
• determine a plane in the light field model by approximating a curvature of a surface of the eye based on depth information from the plenoptic image;
• identify a region of the plane that includes an iris of the eye based on a size of the iris of the eye;
• and
• determine a gaze direction of the eye based on the identified region.