Methods for obtaining normal vector, geometry and material of three-dimensional objects based on neural network

A method for obtaining a normal vector of a three-dimensional object based on a neural network, comprising following steps:

• (1) generating training data,
• wherein said generating the training data comprises:
• obtaining parameters of an acquisition device, the parameters comprising a distance from a light source to an origin of sampling space, an angle from the light source to the origin of the sampling space, a characteristic curve of the light source, a distance from a camera to the origin of the sampling space, an angle from the camera to the origin of the sampling space, and an intrinsic parameter and an extrinsic parameter of the camera;
• and generating, utilizing the parameters, an acquisition result that simulates an actual camera, as training data;
• (2) training the neural network based on the training data obtained in the step (1), wherein the neural network has following characteristics:
• a. input of the neural network being a Lumitexel, wherein an incident light from each light source is incident to a sampling point along an observation direction and then is reflected by the sampling point along an observation direction to form a reflected light having a light intensity, and the Lumitexel denotes the light intensity;
• b. output of the neural network and a regression target being normal vectors corresponding to the Lumitexel input;
• c. a first layer of the neural network comprising a linear fully connected layer, wherein a parameter matrix of the linear fully connected layer is trained by a following formula:
• Wl=fw(Wraw),
• where Wraw denotes a parameter to be trained;
• Wl denotes an illumination matrix with a size of n×m, n is sampling accuracy of the Lumitexel, m denotes a number of illumination patterns;
• and fw denotes a mapping used to transform Wraw, so that the generated illumination matrix is capable of corresponding to a possible luminous intensity of the acquisition device;
• and
• d. a second layer and subsequent layers being non-linear mapping networks, and a length of an output vector of a last layer is 3;
• and after the training, the illumination matrix of the linear fully connected layer of the first layer being taken out;
• and
• (3) generating, by the acquisition device, an illumination pattern based on the illumination matrix taken out in the step (2), and irradiating a target three-dimensional object in sequence, to obtain a set of photos r1, r2 . . . , rm;
• traversing pixels and channels of the photos in turn, taking out pixel values of an i-th row, a j-th column, and a k-th channel from each of the r1, r2, . . . , rm each time, to form a vector a=a1, a2, . . . , am, taking a as an output vector of the linear fully connected layer of the first layer of the neural network, calculating the output vector of the last layer as a normal vector of a surface of the three-dimensional object corresponding to one of the pixels;
• and traversing all of the pixels to obtain a feature map of the normal vector of the surface of the three-dimensional object.