Methods and systems for capturing 3D surface geometry

A system for capturing 3D video data of a dynamic scene, the video data comprising a plurality of frames, the system comprising:

• three light sources, each light source configured to emit light at a different wavelength to the other two sources, the three light sources being configured to illuminate the scene to be captured;
• a first video camera configured to receive from the light sources light which has been reflected from the scene, the first video camera being configured to perform a photometric stereo measurement by isolating light received from each of the light sources and output photometric stereo data relating to the image captured for each of the three light sources;
• a second video camera spatially separate from said first video camera, the first video camera and the second video camera being arranged in a two view stereo arrangement to perform a non-photometric stereo measurement to obtain non-photometric stereo data from the first and second video cameras, the non-photometric stereo data having a lower frequency resolution in 2D Fourier space than the photometric stereo data;
• a processor configured to:
• receive said photometric stereo data from the first video camera and process said photometric stereo data to obtain data relating to a first normal field obtained from the images captured for each of the three light sources;
• receive non-photometric stereo data from the two-view stereo arrangement of the first and second video cameras,
• produce data in the form of a depth map from the non-photometric stereo data, and
• combine the first normal field data with that of the non-photometric data in the form of the depth map by:
• blurring the data relating to the first normal field, using a blurring kernel and wherein the variance of the blurring kernel is σ2 where σ2 is set at a value to reduce flickering of surfaces in the dynamic scene between subsequent frames;
• calculating the rotation of the normals of the first normal field caused due to blurring;
• calculating a further normal field from the non-photometric data that has been blurred by the same kernel;
• and
• applying the calculated rotation of the normals of the first normal field caused due to blurring to the normals of the further normal field, to produce 3D image data of the scene.

A method for producing 3D video data of a dynamic scene the video data comprising a plurality of frames, the method comprising:

• receiving photometric stereo data from a first video camera, said photometric stereo data comprising video data captured by said first camera of the scene illuminated from three different directions, the data being capable of isolation into the image data captured for each illumination direction;
• receiving non-photometric stereo data in the form of a first depth map of the scene obtained by a depth sensor using a non-photometric stereo measurement, wherein the first depth map will have lower frequency resolution in 2D Fourier space than the photometric stereo data;
• analyzing the photometric stereo data received from the first video camera and processing said photometric stereo data to obtain data relating to a first normal field;
• the method further comprising combining the first normal field data with that of the non-photometric data in the form of the first depth map by blurring the data relating to the first normal field, using a blurring kernel and wherein the variance of the blurring kernel is σ2 where σ2 is set at a value to reduce flickering of surfaces in the dynamic scene between subsequent frames;
• calculating the rotation of the normals of the first normal field caused due to blurring;
• calculating a further normal field from the non-photometric data that has been blurred by the same kernel;
• and
• applying the calculated rotation of the normals of the first normal field caused due to blurring to the normals of the further normal field, to produce 3D image data of the scene.