Inferring missing details of a point cloud rendering

A non-transitory computer-readable storage medium having recorded thereon a computer program that when executed by a computer causes the computer to implement a method for forming a dataset configured for learning a neural network architecture, the neural network architecture being configured for inferring missing image details of a point cloud rendering, the method comprising:

• obtaining a 3D mesh scene;
• computing a point cloud representation of the 3D mesh scene;
• generating one or more camera views of the 3D mesh scene and the point cloud representation, each camera view being generated on both the 3D mesh scene and the point cloud representation, a respective generated camera view having a position and/or orientation different from at least another generated camera view;
• for each camera view:
• rendering a viewpoint of the point cloud representation, thereby obtaining an image together with a corresponding depth map,
• rendering a viewpoint of the 3D mesh scene,
• computing another point cloud representation of the viewpoint of the 3D mesh scene, and
• rendering a viewpoint of the other point cloud representation, thereby obtaining another image together with another corresponding depth map;
• obtaining a pair of training samples, each training sample comprising respectively the rendered viewpoint of the point cloud representation and the rendered viewpoint of the other point cloud representation;
• and
• adding the pair of training samples to the dataset.

A system comprising:

• a processor coupled to a memory, the memory having recorded thereon a computer program for forming a dataset configured for learning a neural network architecture, the neural network architecture being configured for inferring missing image details of a point cloud rendering, that when executed by the processor causes the processor to be configured to:
• obtain a 3D mesh scene;
• compute a point cloud representation of the 3D mesh scene;
• generate one or more camera views of the 3D mesh scene and the point cloud representation, each camera view being generated on both the 3D mesh scene and the point cloud representation, a respective generated camera view having a position and/or orientation different from at least another generated camera view;
• for each camera view:
• render a viewpoint of the point cloud representation, thereby obtaining an image together with a corresponding depth map,
• render a viewpoint of the 3D mesh scene,
• compute another point cloud representation of the viewpoint of the 3D mesh scene, and
• render a viewpoint of the other point cloud representation, thereby obtaining another image together with another corresponding depth map;
• obtain a pair of training samples, each training sample comprising respectively the rendered viewpoint of the point cloud representation and the rendered viewpoint of the other point cloud representation;
• and
• add the pair of training samples to the dataset.