Back to results
US 10,705,600 B2Granted
Method and apparatus for detecting and following an eye and/or the gaze direction thereof
AssigneeTobii AB
InventorMarten Skogo
Life of this patent
Estimating the term…
Abstract
An eye-tracking system comprises an illuminator that directs infrared light towards a detection region, a camera configured to generate an image of the detection region and a controller configured to detect a calibration target in the image and to detect or determine changes in the position and/or orientation of the camera relative to the calibration target. The eye-tracking system can be installed in a vehicle.
Claims20
Apparatus for detecting the gaze direction of an eye of a user in a vehicle, the apparatus comprising:
- an illuminator arranged to direct infrared light towards a detection region, the detection region including the eye of the user and a calibration target at a known location in the vehicle;
- a plurality of cameras having at least partially overlapping fields of view in the detection region and configured to generate an image of the detection region which includes the calibration target and the eye of the user;
- a controller configured to detect from the image the calibration target in an overlap region of the image corresponding to the at least partially overlapping fields of view of at least two of the plurality of cameras, and to determine the position and/or orientation of the at least two cameras relative to the calibration target, wherein the controller detects the calibration target from at least two of the plurality of cameras simultaneously;
- a sensor configured to detect a change in the position and/or orientation of at least one of the plurality of cameras with respect to the calibration target, and upon detection of a change in the position and/or orientation, the controller configured to detect again the calibration target at the known location in an overlap region of an image of the detection region and to determine again the position and/or orientation of the at least one of the plurality of cameras relative to the calibration target, wherein the controller detects again the calibration target from at least two of the plurality of cameras simultaneously;
- the controller further configured to detect, based on an image from at least one of the plurality of cameras, a gaze direction of an eye of the user of the vehicle.
2 more elements
A method for calibrating and using an apparatus for detecting the gaze direction of an eye of a user in a vehicle, the method comprising:
- directing infrared light towards a detection region which includes the eye of the user and a calibration target at a known location in the vehicle;
- generating an image of the detection region using a plurality of cameras, the plurality of cameras having at least partially overlapping fields of view in the detection region;
- detecting from the image the calibration target in an overlap region corresponding to the at least partially overlapping fields of view of at least two of the plurality of cameras simultaneously;
- determining the position and/or orientation of the at least two cameras relative to the calibration target;
- sensing a change in the position and/or orientation of at least one of the plurality of cameras with respect to the calibration target, and upon sensing of a change in the position and/or orientation, detecting again the calibration target simultaneously at the known location in an overlap region of an image of the detection region and determining again the position and/or orientation of the at least one of the plurality of cameras relative to the calibration target;
- and
- detecting, based on an image from at least one of the plurality of cameras, a gaze direction of the user of the vehicle.
4 more elements
Apparatus for detecting the gaze direction of an eye of a user in a vehicle, the apparatus comprising:
- an illuminator arranged to direct infrared light towards a detection region, the detection region including the eye of the user and calibration targets at known locations in the vehicle;
- a plurality of cameras responsive to the light in the detection region and configured to generate an image of the detection region, said detection region image including images of calibration targets at the known locations, the plurality of cameras having at least partially overlapping fields of view in an overlap region of the detection region;
- a pattern recognition module to recognize the calibration targets;
- a sensor configured to detect a change in the position and/or orientation of at least one of the plurality of cameras;
- and
- a control system responsive to the pattern recognition module, to the sensor and to the cameras and operating suitable programs to:
- (i) determine simultaneously, for each camera, in two orthogonal directions, pixel coordinates of a center of an image of a first calibration target relative to an origin,
- (ii) determine simultaneously, for each camera, in two orthogonal directions, pixel coordinates of a center of an image of a second calibration target relative to the origin, and
- (iii) utilize, for each camera, the pixel coordinates of the first calibration target and of the second calibration target to determine the position and/or orientation of the at least one of the plurality of cameras relative to the calibration targets,
- (iv) upon the sensor detecting a change in the position and/or orientation of the at least one of the plurality of cameras, with respect to the calibration target repeat steps (i) to (iii), and detecting again the calibration target at the known location in a region of an image of the detection region, and
- (v) detect, based on an image from at least one of the plurality of cameras, a gaze direction of an eye of the user of the vehicle.
8 more elements
Apparatus for detecting the gaze direction of an eye of a user in a vehicle, the apparatus comprising:
- an illuminator arranged to direct infrared light towards a detection region, the detection region including the eye of the user and a calibration target at a known location in the vehicle;
- a plurality of cameras responsive to the light in the detection region and configured to generate an image of the detection region, the plurality of cameras having at least partially overlapping fields of view in an overlap region of the detection region;
- a controller responsive to the cameras and configured to detect a calibration target at the known location in the overlap region of the image and to determine the position and/or orientation of the at least two cameras relative to the calibration target, wherein the controller detects the calibration target from at least two of the plurality of cameras simultaneously;
- a sensor configured to detect a change in the position and/or orientation of at least one of the plurality of cameras with respect to the calibration target, and upon a change the controller configured to detect again the calibration target at the known location in a region of an image of the detection region corresponding to an overlapping detection region of at least two of the plurality of cameras, and to determine the position and/or orientation of the at least two cameras relative to the calibration target, wherein the controller detects again the calibration target from at least two of the plurality of cameras simultaneously;
- the controller further configured to detect, based on an image from at least one of the plurality of cameras, a gaze direction of an eye of the user of the vehicle.
2 more elements
Dependencies are inferred from the claim text.