Eye tracking using image data

A method of processing data in a processing apparatus defining a sequence of images of a person as the person's head moves, to track the eyes of the person in the images, the method comprising processing each of at least some of the images in the sequence by:

• determining image values of pixels located in the image at positions defined by the projection of search bands or points therein from a three-dimensional space into the two-dimensional image, wherein the search bands have different positions in the three-dimensional space, and wherein the projection is dependent upon the position and orientation of the search bands in the three-dimensional space resulting from the processing of the previous image in the sequence to be processed;
• calculating positions for the eyes in the three-dimensional space in dependence upon the determined image values;
• and
• moving the search bands within the three-dimensional space in dependence upon the calculated positions for the eyes, so as to move the search bands for subsequent processing of the next image in the sequence to be processed.

A method of processing data in a processing apparatus to perform eye tracking across a sequence of images of a moving head, the method comprising processing each of at least some of the images in the sequence by:

• identifying pixels in the image corresponding to search regions defined relative to a three-dimensional computer model representative of at least part of a head in a three-dimensional space, the search regions being defined with different positions in a portion of the three-dimensional space corresponding to the eyes and eyebrows of the model, and the identification of pixels being performed in dependence upon data defining a transformation between the three-dimensional space and the two-dimensional image;
• determining image values of the identified pixels;
• generating an image value profile from the image values representative of the variation in image value with variation in search region position;
• and
• comparing the generated image value profile with a reference profile to determine eye positions in the three-dimensional space.

A method of processing data in a processing apparatus defining a sequence of images of a person as the person's head moves, to track the eyes of the person in the images, the method comprising processing each of at least some of the images in the sequence by:

• determining image values of pixels located in the image at positions defined by the projection of search bands or positions therein from a three-dimensional space into the two-dimensional image, wherein the search bands have different positions in the three-dimensional space, and wherein the projection is dependent upon the position and orientation of the search bands in the three-dimensional space resulting from the processing of the previous image in the sequence to be processed;
• generating at least one representative image value for each search band using the determined pixel image values;
• comparing the generated representative image values with reference image values representative of image values for an eye and an eyebrow to calculate positions for the eyes in the three-dimensional space;
• and
• moving the search bands within the three-dimensional space in dependence upon the calculated positions for the eyes, so as to move the search bands for subsequent processing of the next image in the sequence to be processed.

A method of processing data in a processing apparatus to perform eye tracking across a sequence of images of a moving head, the method comprising processing each of at least some of the images in the sequence by:

• identifying pixels in the image in dependence upon search regions defined relative to a three-dimensional computer model representative of at least part of a head in a three-dimensional space, the search regions dividing into parts a portion of the three-dimensional space corresponding to the eyes and eyebrows of the model, and in dependence upon data defining a transformation between the three-dimensional space and the two-dimensional image;
• determining image values of the identified pixels;
• determining which of the search regions project onto dark regions in the image in dependence upon the image values;
• and
• determining eye positions in the three-dimensional space in dependence upon the relative positions of search regions determined to project onto dark regions of the image.

A method of processing data in a processing apparatus defining a sequence of images of a person as the person's head moves, to track the eyes of the person in the images, the method comprising processing each of least some of the images in the sequence by:

• determining the image values of pixels located in the image at positions defined by the projection of search bands or points therein from a three-dimensional space into the two-dimensional image, wherein the search bands have different positions in the three-dimensional space so as to divide into parts a portion of the three-dimensional space, and wherein the projection is dependent upon the position and orientation of the search bands in the three-dimensional space resulting from the processing of the previous image in the sequence to be processed;
• calculating a respective representative image value for each search band using the determined pixel image values;
• calculating positions for the eyes in the three-dimensional space by using the representative values to identify search bands corresponding to dark regions in the image and in dependence upon the relative positions of the identified search bands;
• and
• moving the search bands within the three-dimensional space in dependence upon the calculated positions for the eyes, so as to move the search bands for subsequent processing of the next image in the sequence to be processed.

Image processing apparatus for processing image data of at least some images in a sequence of images of a moving head to perform eye tracking across the sequence, the apparatus comprising:

• a data store configured to store data defining a three-dimensional computer model representative of at least part of a head in a three-dimensional space, and data defining search regions having different positions in a portion of the three-dimensional space corresponding to the eyes and eyebrows of the model;
• a pixel identifier operable to identify pixels in an image corresponding to the search regions, the pixel identifier being operable to identify pixels in dependence upon a transformation between the three-dimensional space and the two-dimensional image;
• an image value determinator operable to determine image values of the identified pixels;
• and
• an eye position determinator operable to determine eye positions in the three-dimensional space in dependence upon the determined image values.

Apparatus for processing image data of each of at least some images in a sequence of images of a person as the person's head moves, to track the eyes of the person in the images, the apparatus comprising:

• a pixel identifier operable to identify pixels located in the image at positions defined by the projection of search bands or points therein from a three-dimensional space into the two-dimensional image, wherein the search bands have different positions in the three-dimensional space, and wherein the projection is dependent upon the position and orientation of the search bands in the three-dimensional space resulting from the processing of the previous image in the sequence to be processed;
• an image value determinator operable to determine image values of the identified pixels;
• an eye position calculator operable to calculate positions for the eyes in the three-dimensional space in dependence upon the determined image values;
• and
• a search band mover operable to move the search bands within the three-dimensional space in dependence upon the calculated positions for the eyes, so as to move the search bands for subsequent processing of the next image in the sequence to be processed.

Apparatus for processing image data of each of at least some images in a sequence of images of a moving head to perform eye tracking across the sequence, the apparatus comprising:

• a data store configured to store data defining a three-dimensional computer model representative of at least part of a head in a three-dimensional space, and data defining search regions having different positions in a portion of the three-dimensional space corresponding to the eyes and eyebrows of the model;
• a pixel identifier operable to identify pixels in the image corresponding to the search regions, the pixel identifier being operable to identify pixels in dependence upon a transformation between the three-dimensional space and the two-dimensional image;
• an image value determinator operable to determine image values of the identified pixels;
• a profile generator operable to generate an image value profile from the image values representative of the variation in image value with variation in search region position;
• and
• a profile comparer operable to compare the generated image value profile with a reference profile to determine eye positions in the three-dimensional space.

Apparatus for processing image data of each of at least some images in a sequence of images of a person as the person's head moves, to track the eyes of the person in the images, the apparatus comprising:

• a pixel identifier operable to identify pixels located in the image at positions defined by the projection of search bands or positions therein from a three-dimensional space into the two-dimensional image, wherein the search bands have different positions in the three-dimensional space, and wherein the projection is dependent upon the position and orientation of the search bands in the three-dimensional space resulting from the processing of the previous image in the sequence to be processed;
• an image value determinator operable to determine image values of the identified pixels;
• a value generator operable to generate at least one representative image value for each search band using the determined pixel image values;
• a comparer operable to compare the generated representative image values with reference image values representative of image values for an eye and an eyebrow to calculate positions for the eyes in the three-dimensional space;
• and
• a search band mover operable to move the search bands within the three-dimensional space in dependence upon the calculated positions for the eyes, so as to move the search bands for subsequent processing of the next image in the sequence to be processed.

Apparatus for processing image data of at least some images in a sequence of images of a moving head to perform eye tracking across the sequence, the apparatus comprising:

• a data store configured to store data defining a three-dimensional computer model representative of at least part of a head in a three-dimensional space, and data defining search regions dividing into parts a portion of the three-dimensional space corresponding to the eyes and eyebrows of the model;
• a pixel identifier operable to identify pixels in the image in dependence upon the search regions and in dependence upon data defining a transformation between the three-dimensional space and the two-dimensional image;
• an image value determinator operable to determine image values of the identified pixels;
• a search region selector operable to determine which of the search regions project onto dark regions in the image in dependence upon the image values;
• and
• an eye position determinator operable to determine eye positions in the three-dimensional space in dependence upon the relative positions of search regions determined to project onto dark regions of the image.

Apparatus for processing image data of each of at least some images in a sequence of images of a person as the person's head moves, to track the eyes of the person in the images, the apparatus comprising:

• a pixel identifier operable to identify pixels located in the image at positions defined by the projection of search bands or points therein from a three-dimensional space into the two-dimensional image, wherein the search bands have different positions in the three-dimensional space so as to divide into parts a portion of the three-dimensional space, and wherein the projection is dependent upon the position and orientation of the search bands in the three-dimensional space resulting from the processing of the previous image in the sequence to be processed;
• an image value determinator operable to determine image values of the identified pixels;
• a value calculator operable to calculate a respective representative image value for each search band using the determined pixel image values;
• an eye position calculator operable to calculate positions for the eyes in the three-dimensional space by using the representative values to identify search bands corresponding to dark regions in the image and in dependence upon the relative positions of the identified search bands;
• and
• a search band mover operable to move the search bands within the three-dimensional space in dependence upon the calculated positions for the eyes, so as to move the search bands for subsequent processing of the next image in the sequence to be processed.

Image processing apparatus for processing image data of at least some images in a sequence of images of a moving head to perform eye tracking across the sequence, the apparatus comprising:

• means for storing data defining a three-dimensional computer model representative of at least part of a head in a three-dimensional space, and data defining search regions having different positions in a portion of the three-dimensional space corresponding to the eyes and eyebrows of the model;
• pixel identifying means for identifying pixels in an image corresponding to the search regions, the pixel identifying means being operable to identify pixels in dependence upon a transformation between the three-dimensional space and the two-dimensional image;
• means for determining image values of the identified pixels;
• and
• means for determining eye positions in the three-dimensional space in dependence upon the determined image values.

Apparatus for processing image data of each of at least some images in a sequence of images of a person as the person's head moves, to track the eyes of the person in the images, the apparatus comprising:

• means for identifying pixels located in the image at positions defined by the projection of search bands or points therein from a three-dimensional space into the two-dimensional image, wherein the search bands have different positions in the three-dimensional space, and wherein the projection is dependent upon the position and orientation of the search bands in the three-dimensional space resulting from the processing of the previous image in the sequence to be processed;
• means for determining image values of the identified pixels;
• means for calculating positions for the eyes in the three-dimensional space in dependence upon the determined image values;
• and
• means for moving the search bands within the three-dimensional space in dependence upon the calculated positions for the eyes, so as to move the search bands for subsequent processing of the next image in the sequence to be processed.

Apparatus for processing image data of each of at least some images in a sequence of images of a moving head to perform eye tracking across the sequence, the apparatus comprising:

• means for storing data defining a three-dimensional computer model representative of at least part of a head in a three-dimensional space, and data defining search regions having different positions in a portion of the three-dimensional space corresponding to the eyes and eyebrows of the model;
• pixel identifying means for identifying pixels in the image corresponding to the search regions, the pixel identifying means being operable to identify pixels in dependence upon a transformation between the three-dimensional space and the two-dimensional image;
• means for determining image values of the identified pixels;
• means for generating an image value profile from the image values representative of the variation in image value with variation in search region position;
• and
• means for comparing the generated image value profile with a reference profile to determine eye positions in the three-dimensional space.

Apparatus for processing image data of each of at least some images in a sequence of images of a person as the person's head moves, to track the eyes of the person in the images, the apparatus comprising:

• means for identifying pixels located in the image at positions defined by the projection of search bands or positions therein from a three-dimensional space into the two-dimensional image, wherein the search bands have different positions in the three-dimensional space, and wherein the projection is dependent upon the position and orientation of the search bands in the three-dimensional space resulting from the processing of the previous image in the sequence to be processed;
• means for determining image values of the identified pixels;
• means for generating at least one representative image value for each search band using the determined pixel image values;
• means for comparing the generated representative image values with reference image values representative of image values for an eye and an eyebrow to calculate positions for the eyes in the three-dimensional space;
• and
• means for moving the search bands within the three-dimensional space in dependence upon the calculated positions for the eyes, so as to move the search bands for subsequent processing of the next image in the sequence to be processed.

Apparatus for processing image data of at least some images in a sequence of images of a moving head to perform eye tracking across the sequence, the apparatus comprising:

• means for storing data defining a three-dimensional computer model representative of at least part of a head in a three-dimensional space, and data defining search regions dividing into parts a portion of the three-dimensional space corresponding to the eyes and eyebrows of the model;
• pixel identifying means for identifying pixels in the image in dependence upon the search regions and in dependence upon data defining a transformation between the three-dimensional space and the two-dimensional image;
• means for determining image values of the identified pixels;
• search region selection means for determining which of the search regions project onto dark regions in the image in dependence upon the image values;
• and
• eye position determining means for determining eye positions in the three-dimensional space in dependence upon the relative positions of search regions determined to project onto dark regions of the image.

Apparatus for processing image data of each of at least some images in a sequence of images of a person as the person's head moves, to track the eyes of the person in the images, the apparatus comprising:

• means for identifying pixels located in the image at positions defined by the projection of search bands or points therein from a three-dimensional space into the two-dimensional image, wherein the search bands have different positions in the three-dimensional space so as to divide into parts a portion of the three-dimensional space, and wherein the projection is dependent upon the position and orientation of the search bands in the three-dimensional space resulting from the processing of the previous image in the sequence to be processed;
• means for determining image values of the identified pixels;
• means for calculating a respective representative image value for each search band using the determined pixel image values;
• means for calculating positions for the eyes in the three-dimensional space by using the representative values to identify search bands corresponding to dark regions in the image and in dependence upon the relative positions of the identified search bands;
• and
• means for moving the search bands within the three-dimensional space in dependence upon the calculated positions for the eyes, so as to move the search bands for subsequent processing of the next image in the sequence to be processed.