Back to results
US 10,591,731 B2Granted
Ocular video stabilization
AssigneeGoogle Inc.
InventorErfan Davami
Life of this patent
Estimating the term…
Abstract
A system and method for ocular stabilization of video images is disclosed. While capturing video images in a forward field of view with a forward-facing video camera of a wearable head-mountable device (HMD), binocular eye-gaze directions of left and right eyes of a user of the HMD may be obtained with an eye-tracking device of the HMD. Based on the obtained binocular eye-gaze directions of left and right eyes of the user of the HMD, convergent gaze directions of the user may be determined as a function of time during an interval concurrent with the capturing of the video images. The captured video images may then be stabilized by compensating for motion of the forward-facing video camera with an intersection of the convergent gaze directions of the user with an image plane of the forward-facing video camera.
Claims14
In a wearable head-mountable device (HMD), a computer-implemented method comprising:
- while capturing video images in an image plane of a forward-facing video camera of and attached to the HMD, obtaining stereoscopic video images of left and right eyes of a user of the HMD with left and right eye-facing video cameras of and attached to the HMD;
- based on the obtained stereoscopic video images of the left and right eyes of the user of the HMD, determining convergent gaze directions of the user in a forward field of view (FOV) as a function of time during an interval concurrent with the capturing of the video images;
- determining a trajectory of intersections of the convergent gaze directions with the image plane as a function of time during the interval;
- and
- shifting the captured video images in the image plane along the trajectory to align the video images with a common point in the image plane at each instant during the interval,
- wherein determining the trajectory of intersections of the convergent gaze directions with the image plane as a function of time during the interval comprises:
- determining the intersection of the convergent gaze directions of the user with the image plane at each of a sequence of two or more times;
- determining planar coordinates in the image plane of each intersection of the sequence;
- and
- specifying one intersection of the sequence as a reference point in the image plane;
- and wherein shifting the captured video images in the image plane along the trajectory to align the video images with the common point in the image plane at each instant during the interval comprises:
- shifting the video image captured at each respective time of the one or more sequence times to align the planar coordinates of the intersection at the respective time with the planar coordinates of the reference point.
9 more elements
In a wearable head-mountable device (HMD), a computer-implemented method comprising:
- while capturing video images in an image plane of a forward-facing video camera of and attached to the HMD, obtaining eye-gaze directions of at least one of a left and a right eye of a user of the HMD with an eye-tracking device of and attached to the HMD;
- based on the obtained eye-gaze directions, determining gaze directions of the user in a forward field of view (FOV) as a function of time during an interval concurrent with the capturing of the video images;
- determining a trajectory of intersections of the gaze directions with an image plane as a function of time during the interval;
- and
- shifting the captured video images in the image plane along the trajectory to align the video images with a common point in the image plane at each instant during the interval,
- wherein determining the trajectory of intersections of the convergent gaze directions with the image plane as a function of time during the interval comprises:
- determining the intersection of the convergent gaze directions of the user with the image plane at each of a sequence of two or more times;
- determining planar coordinates in the image plane of each intersection of the sequence;
- and
- specifying one intersection of the sequence as a reference point in the image plane;
- and wherein shifting the captured video images in the image plane along the trajectory to align the video images with the common point in the image plane at each instant during the interval comprises:
- shifting the video image captured at each respective time of the one or more sequence times to align the planar coordinates of the intersection at the respective time with the planar coordinates of the reference point.
9 more elements
A head-mountable device (HMD) comprising:
- a forward-facing video camera;
- an eye-tracking device;
- a processor;
- and
- memory storing instructions that, when executed by the processor, cause the HMD to carry out operations including:
- while capturing video images in an image plane of the forward-facing video camera, obtaining binocular eye-gaze directions of left and right eyes of a user of the HMD with the eye-tracking device;
- based on the obtained binocular eye-gaze directions of the left and right eyes of the user of the HMD, determining convergent gaze directions of the user in a forward field of view (FOV) as a function of time during an interval concurrent with the capturing of the video images;
- determining a trajectory of intersections of the convergent gaze directions with an image plane as a function of time during the interval;
- and
- shifting the captured video images in the image plane along the trajectory to align the video images with a common point in the image plane at each instant during the interval,
- wherein determining the trajectory of intersections of the convergent gaze directions with the image plane as a function of time during the interval comprises:
- determining the intersection of the convergent gaze directions of the user with the image plane at each of a sequence of two or more times;
- determining planar coordinates in the image plane of each intersection of the sequence;
- and
- specifying one intersection of the sequence as a reference point in the image plane;
- and wherein shifting the captured video images in the image plane along the trajectory to align the video images with the common point in the image plane at each instant during the interval comprises:
- shifting the video image captured at each respective time of the one or more sequence times to align the planar coordinates of the intersection at the respective time with the planar coordinates of the reference point.
14 more elements
Dependencies are inferred from the claim text.