Camera Calibration with Gaze Tracking
AssigneeApple Inc.
InventorsStergios Roumeliotis, Daniel C Byrnes, Lionel E Edwin, Tom Sengelaub
Life of this patent
Estimating the term…
Abstract
Forward and backward facing cameras on a head mounted device may be calibrated to each other using gaze tracking while the eye is directed to an external target that is within view of the forward-facing camera. The use of gaze tracking as a calibration proxy enables camera calibration without equipment other than the head mounted device. Camera calibration enables the head mounted device to determine where in an external scene a user wearing the head mounted device is directing the user's gaze.
Claims20
A head-mounted device, comprising:
- a frame;
- a first camera coupled to the frame and directed towards an external scene;
- a second camera coupled to the frame and directed towards an eye of a user;
- and
- one or more computing devices configured to:
- generate world information based on one or more images of the external scene from the first camera, wherein the world information comprises a position of an external target in the external scene;
- determine, based on the world information, a camera vector which corresponds to a direction from the first camera to the external target;
- generate gaze information based on one or more other images of the eye from the second camera, wherein the gaze information comprises a gaze direction of the eye, wherein the gaze direction is associated with the external target;
- determine, based on the gaze information, a gaze vector which corresponds to the gaze direction;
- and
- determine, using the gaze vector and the camera vector, calibration information for calibrating the first camera and the second camera to each other.
A method, comprising:
- receiving one or more images of an external scene from a first camera of a head-mounted device;
- generating world information based on the one or more images of the external scene from the first camera, wherein the world information comprises a position of an external target in the external scene;
- determining, based on the world information, a camera vector which corresponds to a direction from the first camera to the external target;
- receiving one or more other images of an eye from a second camera of the head-mounted device;
- generating gaze information based on the one or more other images of the eye from the second camera, wherein the gaze information comprises a gaze direction of the eye, wherein the gaze direction is associated with the external target;
- determining, based on the gaze information, a gaze vector which corresponds to the gaze direction;
- and
- determining, using the gaze vector and the camera vector, calibration information for calibrating the first camera and the second camera to each other.
A non-transitory, computer-readable storage device storing program instructions that, when executed on or across one or more processors, cause the one or more processors to:
- receive one or more images of an external scene from a first camera of a head-mounted device;
- generate world information based on the one or more images of the external scene from the first camera, wherein the world information comprises a position of an external target in the external scene;
- determine, based on the world information, a camera vector which corresponds to a direction from the first camera to the external target;
- receive one or more other images of an eye from a second camera of the head-mounted device;
- generate gaze information based on the one or more other images of the eye from the second camera, wherein the gaze information comprises a gaze direction of the eye, wherein the gaze direction is associated with the external target;
- determine, based on the gaze information, a gaze vector which corresponds to the gaze direction;
- and
- determine, using the gaze vector and the camera vector, calibration information for calibrating the first camera and the second camera to each other.
Dependencies are inferred from the claim text.