Method of determining illumination pattern for three-dimensional scene and method and apparatus for modeling three-dimensional scene

A method of modeling a three-dimensional (3D) scene, the method comprising:

• obtaining illumination patterns, corresponding to a 3D target object, by using a trained neural network;
• capturing a target scene comprising the 3D target object by using the illumination patterns;
• and
• modeling a 3D scene corresponding to the target scene by restoring, based on the illumination patterns, a surface normal of the 3D target object using a photometric stereo technique,
• wherein the obtaining of the illumination patterns comprises:
• constructing a dataset by estimating a first surface normal vector of the 3D target object from a first image obtained by capturing the 3D target object of which surface normal information is known, the dataset comprising basis images of the 3D target object;
• generating simulation images in which virtual illumination patterns, obtained based on a combination of the basis images, are applied to the 3D target object:
• estimating a second surface normal vector of the 3D target object, by reconstructing a surface normal using a photometric stereo technique based on the virtual illumination patterns and simulation images corresponding to the virtual illumination patterns;
• and
• training the neural network to determine the illumination patterns based on a difference between the first surface normal vector and the second surface normal vector.

An apparatus for modeling a three-dimensional (3D) scene, the apparatus comprising:

• a communication interface configured to receive illumination patterns corresponding to a 3D target object;
• a camera configured to capture a target scene comprising the 3D target object by using the illumination patterns;
• and
• a processor configured to model a 3D scene corresponding to the target scene by restoring, based on the illumination patterns, a surface normal of the 3D target object using a photometric stereo technique,
• wherein the received illumination patterns are determined by:
• constructing a dataset by estimating a first surface normal vector of the 3D target object from a first image obtained by capturing the 3D target object of which surface normal information is known, the dataset comprising basis images of the 3D target object;
• generating simulation images in which virtual illumination patterns, obtained based on a combination of the basis images, are applied to the 3D target object;
• estimating a second surface normal vector of the 3D target object, by reconstructing a surface normal using a photometric stereo technique based on the virtual illumination patterns and simulation images corresponding to the virtual illumination patterns;
• and
• training a neural network to determine the illumination patterns based on a difference between the first surface normal vector and the second surface normal vector.