Method and system for stereo gaze tracking
AssigneeIntuitive Surgical Operations, Inc.
InventorsWenyi Zhao, Brandon D. Itkowitz, George Mylonas, Gung-Zhong Yang
Life of this patent
Estimating the term…
Abstract
Stereo gaze tracking estimates a 3-D gaze point by projecting determined right and left eye gaze points on left and right stereo images. The determined right and left eye gaze points are based on one or more tracked eye gaze points, estimates for non-tracked eye gaze points based upon the tracked gaze points and image matching in the left and right stereo images, and confidence scores indicative of the reliability of the tracked gaze points and/or the image matching.
Claims20
A method for stereo gaze tracking, the method comprising:
- tracking, with a tracking system having a single-eye tracker, a gaze point of a first eye of a user on a first one of right and left two-dimensional display screens of a stereo viewer, wherein the right and left two-dimensional display screens respectively display information of right and left stereo images of a scene captured by a stereoscopic camera, wherein the right and left two-dimensional display screens are visible to the user through right and left eyepieces, respectively;
- and
- estimating, by a processor, a gaze point of a second eye of the user on a second one of the right and left two-dimensional display screens by processing information, received from the tracking system, describing the tracked gaze point of the first eye only and information of the right and left stereo images of the scene.
A method for stereo gaze tracking of a user, the method comprising:
- tracking, with a tracking system, a gaze point of a first eye of the user on a first one of right and left two-dimensional display screens of a stereo viewer, wherein the right and left two-dimensional display screens respectively display information of right and left stereo images of a scene captured by a stereoscopic camera;
- estimating, by a processor, a gaze point of a second eye of a user on a second one of the right and left two-dimensional display screens by processing information of the tracked gaze point of the first eye only and information of the right and left stereo images of the scene, wherein the processing of the information of the tracked gaze point of the first eye and the information of the right and left stereo images of the scene comprises:
- determining one or more of a global offset of the right and left stereo images, a region offset of corresponding regions of the right and left stereo images using the global offset and the tracked gaze point of the first eye, or a feature offset of corresponding features within the corresponding regions of the right and left stereo images;
- and
- estimating the gaze point of the second eye of the user on the second one of the right and left two-dimensional display screens by using one of the region offset, the feature offset, an interpolation of previously determined and prior-in-time gaze points of the second eye of the user, or the global offset according to a reliability of at least one of the determinations and an availability of the previously determined and prior-in-time gaze points of the second eye of the user.
A system comprising:
- a stereo viewer having right and left two-dimensional display screens disposed so as to align with right and left eyepieces that align with right and left eyes of a user, wherein the right and left two-dimensional display screens respectively display information of right and left stereo images of a scene captured by a stereoscopic camera;
- a first tracking system having a first single-eye tracker configured to track a gaze point of a first eye of the user through one of the right and left eyepieces on a first one of the right and left two-dimensional display screens;
- and
- a processor configured to estimate a gaze point of a second eye of the user on a second one of the right and left two-dimensional display screens by processing information of the tracked gaze point of the first eye only and the information of the right and left stereo images of the scene.
Dependencies are inferred from the claim text.