Facial image processing system
AssigneeSeeing Machines Pty Ltd
InventorsTimothy Edwards, Jochen Heinzmann, Sebastien Rougeaux, Alex Zelinsky
Life of this patent
Estimating the term…
Abstract
A method of determining an eye gaze direction of an observer is disclosed comprising the steps of: (a) capturing at least one image of the observer and determining a head pose angle of the observer; (b) utilizing the head pose angle to locate an expected eye position of the observer, and (c) analyzing the expected eye position to locate at least one eye of the observer and observing the location of the eye to determine the gaze direction.
Claims31
A method of determining a head-pose measurement of a person, said head pose measurement including a rotational displacement and translation displacement, said method including the steps of:
- (a) capturing images of the face and head of said person from at least two viewpoints;
- (b) computing an initial estimate of head-pose;
- (c) computing the three dimensional location of two or more facial features;
- and
- (d) determining a rotational displacement and translation displacement which best match a head-model to the computed location of said two or more facial features.
A method of passively determining the direction of eye gaze of a person, said method including the steps of:
- (a) capturing images of the face and head of said person from at least two viewpoints;
- (b) localising the face within two or more of the captured images;
- (c) computing an initial estimate of head-pose;
- (d) identifying feature search regions, based on said initial estimate of head pose, within each captured image that may contain an identifiable facial feature;
- (e) measuring the position of two or more facial features within two or more respective feature search regions in each of said captured images;
- (f) computing the three-dimensional location of each of said two or more facial features;
- (g) determining a head pose measurement, said head pose measurement including a rotational displacement and translation displacement which best match a head-model to the computed location of said two or more facial features;
- (h) using said head pose measurement to determine the location the centre of one eyeball;
- (i) using said head pose measurement to find an eye region corresponding with said eyeball, within each captured image, and determining the location of the centre of the iris of said eye;
- (j) using the determined position of said centre of said eyeball and the determined location of the centre of the iris of said eye, to calculate an eye-gaze direction for said eye.
A method of determining the eye closure amount of an eye located on a person's face, said eye having an eyeball, a top eyelid, and a bottom eyelid, said top and bottom eyelids meeting at a pair of eye corners;
- wherein eye closure amount is defined as the ratio of eye corner separation to eyelid distance, said method including the steps of:
- (a) capturing images of the face and head of said person from at least two viewpoints;
- (b) determining a head pose measurement, from said captured images;
- (c) using said head pose measurement to determine an eye region within said captured image;
- (d) determining the location of said eye corners within said eye region;
- (e) fitting at least two oriented edge detectors to said top eyelid and to said bottom eyelid, such that one of said oriented edge detectors of said top eyelid intersects with one oriented edge detectors of said bottom eyelid at each eye corner;
- (f) approximating the position of said top eyelid, and said bottom eyelid, by fitting a parabola to said at least two oriented edge detectors corresponding to each eyelid, such that each parabola is tangential to the at least two oriented edge detectors of said parabola's respective eyelid;
- (g) determining said eye corner separation by calculating the distance between said eye corners;
- (h) determining the maximum eyelid distance by calculating the maximum distance between said parabolas on the portion of said parabolas between said eye corners, in a direction perpendicular to a line joining the eye corners;
- (i) calculating the eye closure amount by taking the ratio of the determined eye corner separation to maximum eyelid distance.
A method of determining an eye gaze direction of an observer comprising the steps of:
- (a) capturing at least one image of the observer and determining a head pose angle of said observer;
- (b) utilising said head pose angle to locate an expected eye position of said observer;
- and
- (c) analysing said expected eye position to locate at least one eye of said observer, and observing the location of the eye to determine an eye gaze direction.
A method as claimed in any previous claim wherein said method further includes determining the three dimensional position of a centre of rotation of an observer's eye.
A method as claimed in any previous claim wherein said method further determines eye closure and blinking.
A method of detecting a flexible feature on a person's face, said method including the step of
- comparing the variance of said feature's position from an expected position of said facial feature, based on a head-pose measurement, with a threshold variance to determine whether the feature is flexible.
A method of analysis of a person performing a task wherein the method of any one of the previous claims is used to analyse an action of said subject.
A method of animating an image, wherein the movement of a subject is tracked using a method according to any one of the previous claims, and wherein said image is animated in response to said movement.
A method for measuring operator fatigue including the step of:
- tracking any one or more of:
- head-pose;
- eye gaze;
- eye closure amount;
- or blink rate of the operator.
An apparatus when implementing a method of any previous claim.
A method of detecting eye, gaze substantially as hereinbefore described with reference to the accompanying drawings.
A method of detecting head-pose substantially as hereinbefore described with reference to the accompanying drawings.
A method of blink detection substantially as hereinbefore described with reference to the accompanying drawings.
A method of determining eye closure amount substantially as hereinbefore described with reference to the accompanying drawings.
Dependencies are inferred from the claim text.