Semantically-guided planar three dimensional (3d) reconstruction

An apparatus for image processing, the apparatus comprising:

• at least one memory;
• and
• at least one processor coupled to the at least one memory and configured to:
• generate, based on a first depth map corresponding to a scene and a first semantic segmentation map of the scene, a plurality of first blocks associated with a planar surface within the scene;
• determine, based on the plurality of first blocks, a representation of the planar surface;
• replace, based on the representation of the planar surface, first depths for pixels associated with the planar surface in the first depth map with second depths for the pixels associated with the planar surface from a prior depth map corresponding to the scene to generate an updated depth map corresponding to the scene;
• and
• generate, based on the updated depth map corresponding to the scene and the first semantic segmentation map of the scene, a plurality of second blocks associated with the planar surface within the scene.