Back to results
US 2022/0027720 A1Published application
Method to parameterize a 3d model
AssigneeItseez3D, Inc.
InventorIlya LYSENKOV
Life of this patent
Estimating the term…
Abstract
The present invention provides a method to produce a 3D model of a person or an object from just one or several image. The method uses a neural network that is trained on pairs of 3D models of human heads and their frontal images, and then, given an image, infers a 3D model.
Claims6
A method of neural network learning of 3D models of human heads comprising the steps of:
- a) providing at least two training 3D models produced by scanning or modeling representative human heads;
- b) mapping each training 3D model to a pair of target images comprising an Is image that describes the model shape, and an It image that describes the model texture;
- c) parameterizing each 3D model by representing each of the 3D model as a standard triangulated mesh, establishing a correspondence map between each 3D model and the Is image by using the correspondence of each pixel of Is within the standard triangulated mesh to a 3D point p on the surface of the 3D model, and optimizing the cost function to produce a mapping between each 3D model and Is image;
- d) rendering a frontal image of each parameterized 3D model, detecting facial features in the frontal images, and applying a 2D affine transformation to the frontal images in order to make the coordinates of the facial features as close to the average position of facial features for all the parameterized 3D models to produce frontal images of the representative heads;
- e) training a deep learning network on the pair of target images Is and It, and the frontal images of the representative human heads;
- and
- f) using the trained deep learning network to predict the target images Is and It from an input image that contains a frontal photo of a human head.
4 more elements
Dependencies are inferred from the claim text.