Computer application method and apparatus for generating three-dimensional face model, computer device, and storage medium
AssigneeTencent Technology (Shenzhen) Company Limited
InventorsYajing Chen, Yibing Song, Yonggen Ling, Linchao Bao, Wei Liu
Life of this patent
Estimating the term…
Abstract
A computer application method for generating a three-dimensional (3D) face model is provided, performed by a face model generation model running on a computer device, the method including: obtaining a two-dimensional (2D) face image as input to the face model generation model; extracting global features and local features of the 2D face image; obtaining a 3D face model parameter based on the global features and the local features; and outputting a 3D face model corresponding to the 2D face image based on the 3D face model parameter.
Claims17
A method for generating a three-dimensional (3D) face model, performed by a face model generation network running on a computer device, wherein the face model generation network comprises an encoder, the encoder comprising a convolutional neural network (CNN) with a plurality of convolutional layers, the method comprising:
- obtaining a two-dimensional (2D) face image as input to the face model generation network;
- encoding the 2D face image based on the plurality of convolutional layers of the encoder, to obtain global feature vectors of the 2D face image, the global feature vectors representing global features of the 2D face image;
- obtaining a central position of a landmark of the 2D face image;
- extracting, based on the central position, partial features as local features of the 2D face image from features obtained by at least one target convolutional layer in the plurality of convolutional layers, wherein the at least one target convolutional layer corresponds to a target size;
- obtaining a 3D face model parameter based on the global features and the local features;
- and
- outputting a 3D face model corresponding to the 2D face image based on the 3D face model parameter.
A method for training a face model generation network, the face model generation network being used for generating a 3D face model based on a 2D face image, wherein the face model generation network comprises an encoder, the encoder comprising a convolutional neural network (CNN) with a plurality of convolutional layers, the method comprising:
- obtaining a plurality of 2D face image samples;
- invoking an initial network comprising a network parameter, and inputting the plurality of 2D face image samples into the initial network;
- using the initial network to perform for each 2D face image sample in the plurality of 2D face image samples:
- encoding the 2D face image sample based on the plurality of convolutional layers of the encoder, to obtain global feature vectors of the 2D face image sample, the global feature vectors representing global features of the 2D face image sample;
- obtaining a central position of a landmark of the 2D face image sample;
- and
- extracting, based on the central position, partial features as the local features of the 2D face image sample from features obtained by at least one target convolutional layer in the plurality of convolutional layers,
- wherein the at least one target convolutional layer corresponds to a target size;
- obtaining a 3D face model parameter based on the global features and the local features;
- outputting a 3D face model corresponding to the 2D face image sample based on the 3D face model parameter;
- projecting the 3D face model, to obtain a 2D face image corresponding to the 3D face model;
- obtaining a similarity between the 2D face image and the 2D face image sample;
- and
- adjusting the network parameter of the initial network based on the similarity until a target condition is met, to obtain the face model generation network.
A device for generating a 3D face model using a face model generation network running on the device, wherein the face model generation network comprises an encoder, the encoder comprising a convolutional neural network (CNN) with a plurality of convolutional layers, the device comprising a memory for storing computer instructions and a processor in communication with the memory, wherein, when the processor executes the computer instructions, the processor is configured to cause the device to:
- obtain a two-dimensional (2D) face image as input to the face model generation network;
- encode the 2D face image based on the plurality of convolutional layers of the encoder, to obtain global feature vectors of the 2D face image, the global feature vectors representing global features of the 2D face image;
- obtain a central position of a landmark of the 2D face image;
- extract, based on the central position, partial features as local features of the 2D face image from features obtained by at least one target convolutional layer in the plurality of convolutional layers, wherein the at least one target convolutional layer corresponds to a target size;
- obtain a 3D face model parameter based on the global features and the local features;
- and
- output a 3D face model corresponding to the 2D face image based on the 3D face model parameter.
Dependencies are inferred from the claim text.