Fast and precise object alignment and 3D shape reconstruction from a single 2D image
AssigneeOhio State Innovation Foundation
InventorAleix Martinez
Life of this patent
Estimating the term…
Abstract
The innovation describes and discloses systems and methods related to deep neural networks employing machine learning to detect item 2D landmark points from a single image, such as those of an image of a face, and to estimate their 3D coordinates and shape rapidly and accurately. The system also provides for mapping by a feed-forward neural network that defines two criteria, one to learn to detect important shape landmark points on the image and another to recover their depth information. An aspect of the innovation may utilize camera models in a data augmentation approach that aids machine learning of a complex, non-linear mapping function. Other augmentation approaches are also considered.
Claims22
A computer-implemented method for mapping a computerized 2D image into a 3D shape comprising:
- applying machine learning to a predetermined sample size of computerized 2D images, that application developing two criteria, a detection criterion and a transform criterion for a multi-layer deep neural network (“DNN”),
- receiving a non-sample computerized 2D image through a detector of the DNN that has been trained by the application of machine learning,
- regressing, with the detection criterion, the non-sample image that yields detected landmark points through the DNN
- recovering, with the transform criterion, depth information from image attributes;
- and
- mapping, with the transform criterion, the landmark points through linear or nonlinear functions of the DNN that yield a 3D shape corresponding to the non-sample image.
A Deep Neural Network (DNN) system for mapping a computerized 2D image into a 3D shape comprising:
- a machine learning component that receives a predetermined sample size of computerized 2D images and that develops a detection criterion and a transform criterion,
- a detector for receiving a non-sample computerized 2D image, the detector having been trained by the machine learning component;
- and
- a functional mapping component that
- regresses and detects, with the detection criterion, landmark points of the non-sample image,
- recovers, with the transform criterion, depth information from attributes of the non-sample image, and
- maps, with the transform criterion, the landmark points and the recovered depth information into a 3D shape that corresponds to the non-sample image.
Dependencies are inferred from the claim text.