3D reconstruction from images

A non-transitory computer readable storage medium having recorded thereon a computer program having instructions for performing a computer-implemented method of 3D reconstruction of at least one real object comprising an assembly of parts, the method comprising:

• obtaining a neural network configured for generating a 3D primitive CAD object based on an input depth image, the 3D primitive CAD object being defined by sweeping a section along a guide curve, the section being a polygon, a rounded polygon, or a set of one or more curves which forms a closed region, the guide curve being straight line or a continuous curve;
• obtaining (i) a color or grayscale photograph displaying a real world scene including the real object and (ii) a depth image representing the real object;
• segmenting the depth image based at least on the color or grayscale photograph, each segment representing at most a respective part of the assembly;
• obtaining a plurality of pieces of data each representing a respective segment, the plurality of pieces of data comprising at least two different pieces of data;
• applying the neural network to each piece of data separately, each separate application of the neural network resulting in a respective single 3D primitive CAD object;
• and
• after applying the neural network to each piece of data separately and thereby obtaining multiple 3D primitive CAD objects, combining the 3D primitive CAD objects obtained from each segment in order to construct the 3D reconstruction of the real object.

A system comprising:

• a processor coupled to a memory, the memory having recorded thereon a computer program having instructions for performing 3D reconstruction of at least one real object comprising an assembly of parts that when executed by the processor causes the processor to be configured to:
• obtain a neural network configured for generating a 3D primitive CAD object based on an input depth image, the 3D primitive CAD object being defined by sweeping a section along a guide curve, the section being a polygon, a rounded polygon, or a set of one or more curves which forms a closed region, the guide curve being straight line or a continuous curve;
• obtain (i) a color or grayscale photograph displaying a real world scene including the real object and (ii) a depth image representing the real object;
• segment the depth image based at least on the color or grayscale photograph, each segment representing at most a respective part of the assembly;
• obtain a plurality of pieces of data each representing a respective segment, the plurality of pieces of data comprising at least two different pieces of data;
• apply the neural network to each piece of data separately, each separate application of the neural network resulting in a respective single 3D primitive CAD object;
• and
• after applying the neural network to each piece of data separately and thereby obtaining multiple 3D primitive CAD objects, combining the 3D primitive CAD objects obtained from each segment in order to construct the 3D reconstruction of the real object.