US 12,586,412 B2Granted

Using two-dimensional images and machine learning to identify information pertaining to eye shape

AssigneeLashify, Inc.

InventorsSahara Lotti, Jesse Chang

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method for training a machine learning model using information pertaining to an eye area of a human face, the method includes generating training data for the machine learning model. Generating the training data includes generating a first training input, the first training input including information representing 2D images of human faces, and generating a first target output for the training input. The first target output identifies, for each of the 2D images of human faces, information identifying an eye shape category of multiple eye shape categories, the eye shape category corresponding to a shape of an eye represented in the respective 2D image. The method further includes providing the training data to train the machine learning model on (i) a set of training inputs including the first training input, and (ii) a set of target outputs including the first target output.

Claims25

Dependent claims

A method for training a machine learning model using information pertaining to an eye area of a human face, the method comprising:

generating training data for the machine learning model, wherein generating the training data comprises:
generating a first training input, the first training input comprising information representing two-dimensional (2D) images of human faces;
generating a second training input, the second training input comprising information identifying relationships between one or more landmarks on three-dimensional (3D) models of human faces corresponding to the 2D images of human faces, wherein the second training input comprises at least a first landmark relationship representing an upper eyelid area, wherein the first landmark relationship is defined at least by a line between an inner eye corner of a first eye and an outer eye corner of the first eye;
providing the training data to train the machine learning model on a set of training inputs comprising the first training input and the second training input;
and
obtaining from the machine learning model a first training output of a set of training outputs based on the set of training inputs, wherein the first training output identifies, for each of the 2D images of human faces, information identifying an eye shape category describing a dominant feature of an eye represented in the respective 2D image, wherein the eye shape category comprises a computer-derived eye shape category that is generated using unsupervised learning, wherein a second training output comprises information identifying an eye shape sub-category describing a variation of the dominant feature of the eye represented in the respective 2D image.

3 more elements

A method for using a trained machine learning model using information pertaining to an eye area of a human face, the method comprising:

providing to the trained machine learning model a first input comprising two-dimensional (2D) image data representing a 2D image of a face of a subject, a second input comprising information identifying geometric data representing one or more relationships between two or more points of the eye area of the human face, wherein the eye area comprises an eye and an eyelid, and a third input comprising a first landmark relationship between one or more landmarks on a three-dimensional (3D) model of the human face of the subject corresponding to the 2D image of the face of the subject, the first landmark relationship representing an upper eyelid area, wherein the first landmark relationship is defined at least by a line between an inner eye corner of the eye and an outer eye corner of the eye;
and
obtaining, from the trained machine learning model, one or more outputs identifying (i) an eye shape category of a plurality of eye shape categories, the eye shape category corresponding to a shape of an eye represented in the respective 2D image, and (ii) a level of confidence that the eye shape category corresponds to an actual eye shape of the subject represented in the 2D image.

A system comprising:

a memory;
and
one or more processing devices communicatively coupled to the memory, the one or more processing devices to:
provide to a trained machine learning model a first input comprising two-dimensional (2D) image data representing a 2D image of a face of a subject, and a second input comprising information identifying geometric data representing one or more relationships between two or more points of an eye area of the face of the subject, wherein the eye area comprises a first eye and a first eyelid, and a third input comprising a first landmark relationship between one or more landmarks on a three-dimensional (3D) model of the face of the subject corresponding to the 2D image of the face of the subject, the first landmark relationship representing an upper eyelid area, wherein the first landmark relationship is defined at least by a line between an inner eye corner of the first eye and an outer eye corner of the first eye;
and
obtain, from the trained machine learning model, one or more outputs identifying (i) an eye shape category of a plurality of eye shape categories, the eye shape category corresponding to a shape of an eye represented in the respective 2D image, and (ii) a level of confidence that the eye shape category corresponds to an actual eye shape of the subject represented in the 2D image.

3 more elements

Dependencies are inferred from the claim text.