Lossy facial expression training data pipeline
AssigneeBuglife, Inc.
InventorsDaniel DeCovnick, David Schukin
Life of this patent
Estimating the term…
Abstract
Systems and methods allow a data labeler to identify an expression in an image of a labelee's face without being provided with the image. In one aspect, the image of the labelee's face is analyzed to identify facial landmarks. A labeler is selected from a database who has similar facial characteristics to the labelee. A geometric mesh is built of the labeler's face and the geometric mesh is deformed based on the facial landmarks identified from the image of the labelee. The labeler may identify the facial expression or emotion of the geometric mesh.
Claims20
A method for labeling training data for a machine learning algorithm comprising:
- capturing, on a first electronic device, an image of a first face of a labelee;
- analyzing the first face to identify first landmarks of first features of the first face and storing the first landmarks in a database;
- capturing, on a plurality of second electronic devices, images of faces of labelers;
- analyzing the images of faces of labelers to identify second landmarks and storing the second landmarks in the database;
- selecting a labeler having facial features similar to the first face based on the stored first landmarks and second landmarks;
- generating a three-dimensional mesh of a face of the selected labeler, the three-dimensional mesh textured with an image of the selected labeler;
- identifying third landmarks of the three-dimensional mesh that correspond to the first landmarks;
- manipulating positions of the third landmarks based on positions of the first landmarks;
- displaying the three-dimensional mesh to the selected labeler;
- prompting the selected labeler to input an expression or emotion displayed on the three-dimensional mesh;
- associating the input expression or emotion with the image of the first face to generate training data for a machine learning model for expression or emotion identification.
A non-transitory computer-readable medium comprising instructions for labeling training data for a machine learning algorithm, the non-transitory computer-readable medium comprising instructions for:
- capturing, on a first electronic device, an image of a first face of a labelee;
- analyzing the first face to identify first landmarks of first features of the first face and storing the first landmarks in a database;
- capturing, on a plurality of second electronic devices, images of faces of labelers;
- analyzing the images of faces of labelers to identify second landmarks and storing the second landmarks in the database;
- selecting a labeler having facial features similar to the first face based on the stored first landmarks and second landmarks;
- generating a three-dimensional mesh of a face of the selected labeler, the three-dimensional mesh textured with an image of the selected labeler;
- identifying third landmarks of the three-dimensional mesh that correspond to the first landmarks;
- manipulating positions of the third landmarks based on positions of the first landmarks;
- displaying the three-dimensional mesh to the selected labeler;
- prompting the selected labeler to input an expression or emotion displayed on the three-dimensional mesh;
- associating the input expression or emotion with the image of the first face to generate training data for a machine learning model for expression or emotion identification.
Dependencies are inferred from the claim text.