System and method for scene reconstruction with plane and surface reconstruction

An apparatus, comprising:

• a display;
• and
• a processor coupled to the display and configured to:
• segment a scene of an image frame into regions;
• divide the regions into one or more plane regions and one or more non-plane surface regions;
• generate reconstructed planes by performing plane reconstruction based on the one or more plane regions using a first reconstruction approach;
• generate reconstructed surfaces by performing surface reconstruction based on the one or more non-plane surface regions using a second reconstruction approach that is different from the first reconstruction approach;
• and
• create a 3D scene reconstruction by integrating the reconstructed planes and the reconstructed surfaces.

A non-transitory computer-readable medium containing instructions that, when executed by a processor, cause the processor to:

• segment a scene of an image frame into regions;
• divide the regions into one or more plane regions and one or more non-plane surface regions;
• generate reconstructed planes by performing plane reconstruction based on the one or more plane regions using a first reconstruction approach;
• generate reconstructed surfaces by performing surface reconstruction based on the one or more non-plane surface regions using a second reconstruction approach that is different from the first reconstruction approach;
• and
• create a 3D scene reconstruction by integrating the reconstructed planes and the reconstructed surfaces.