Eye tracking using low resolution images
AssigneeApple Inc.
InventorTom Sengelaub
Life of this patent
Estimating the term…
Abstract
Low-power eye tracking for detecting position and movements of a user's eyes in a head-mounted device (HMD). An eye tracking system for an HMD may include eye tracking cameras. The eye tracking cameras may be used to capture low-resolution frames between capturing high-resolution frames, for example by binning pixels on the camera sensor or by capturing horizontal and vertical stripes or lines of pixels on the camera sensor rather than entire frames. This low-resolution information may be used to track relative movement of the user's eyes with respect to the device in intervals between the processing of full, high-resolution frames captured by the eye tracking cameras.
Claims16
A system, comprising:
- a head-mounted device (HMD) configured to display visual content for viewing by a user, wherein the HMD comprises at least one camera configured to switch between a high-resolution mode and a low-resolution mode to respectively capture high-resolution images of the user's eyes and low-resolution images of the user's eyes at a frame rate of N frames per second;
- and
- a controller comprising one or more processors configured to iteratively perform:
- process a high-resolution image captured by the at least one camera in the high-resolution mode using three-dimensional (3D) image processing techniques to track movement of the user's eyes with respect to the HMD;
- signal the at least one camera to switch to the low-resolution mode;
- and
- process one or more low-resolution images subsequently captured by the at least one camera in the low-resolution mode using two-dimensional (2D) image processing techniques to track movement of the user's eyes with respect to the HMD in intervals between high-resolution frames captured by the at least one camera and processed by the controller.
A method, comprising:
- performing, by a controller comprising one or more processors:
- processing a high-resolution image captured by at least one camera of a head-mounted device (HMD) in high-resolution mode using three-dimensional (3D) image processing techniques to track movement of the user's eyes with respect to the HMD;
- signaling the at least one camera to switch to low-resolution mode;
- and
- processing one or more low-resolution images subsequently captured by the at least one camera in the low-resolution mode using two-dimensional (2D) image processing techniques to track movement of the user's eyes with respect to the HMD in intervals between high-resolution frames captured by the at least one camera and processed by the controller.
One or more non-transitory computer-readable storage media storing program instructions that when executed on or across one or more processors cause the one or more processors to:
- process a high-resolution image captured by at least one camera of a head-mounted device (HMD) in high-resolution mode using three-dimensional (3D) image processing techniques to track movement of the user's eyes with respect to the HMD;
- signaling the at least one camera to switch to low-resolution mode;
- and
- process one or more low-resolution images subsequently captured by the at least one camera in the low-resolution mode using two-dimensional (2D) image processing techniques to track movement of the user's eyes with respect to the HMD in intervals between high-resolution frames captured by the at least one camera and processed by the controller.
Dependencies are inferred from the claim text.