Three dimensional scene reconstruction based on contextual analysis

An electronic system for 3-Dimensional (3D) scene reconstruction, the system comprising:

• a 3D reconstruction circuit to receive a plurality of 3D image frames of a scene, each frame comprising a red-green-blue (RGB) image frame comprising color pixels and a depth map frame comprising depth pixels, wherein each of the 3D image frames is associated with a pose of a depth camera that generated the 3D image frames;
• the 3D reconstruction circuit further to project the depth pixels into points in a global coordinate system based on the camera pose and accumulate the projected points into a 3D reconstruction of the scene;
• an object detection circuit to detect objects for each 3D image frame, based on the camera pose, the 3D reconstruction, the RGB image frame and the depth map frame;
• an object recognition circuit to classify the detected objects into one or more object classes;
• and
• a context based fusion circuit to group a plurality of instances of objects in one of the object classes based on a measure of similarity of features between the object instances and to combine point clouds associated with each of the grouped object instances to generate a fused object.

At least one non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, result in the following operations for 3-Dimensional (3D) scene reconstruction, the operations comprising:

• receiving a plurality of 3D image frames of a scene, each frame comprising a red-green-blue (RGB) image frame comprising color pixels and a depth map frame comprising depth pixels, wherein each of the 3D image frames is associated with a pose of a depth camera that generated the 3D image frames;
• projecting the depth pixels into points in a global coordinate system based on the camera pose;
• accumulating the projected points into a 3D reconstruction of the scene;
• detecting objects for each 3D image frame, based on the camera pose, the 3D reconstruction, the RGB image frame and the depth map frame;
• classifying the detected objects into one or more object classes;
• grouping a plurality of instances of objects in one of the object classes based on a measure of similarity of features between the object instances;
• and
• combining point clouds associated with each of the grouped object instances to generate a fused object.