Recognition-based object segmentation of a 3-dimensional image

A system for 3-Dimensional (3D) segmentation of objects, the system comprising:

• an object detection circuit to:
• detect an object in each of a plurality of 3D image frames of a scene based on object recognition, wherein the plurality of 3D image frames are captured by a depth camera, each of the 3D image frames being associated with a pose of the depth camera;
• associate a label with the detected object, the label generated by the object detection circuit based on the object recognition;
• and calculate a 2-Dimensional (2D) bounding box containing the detected object and a 3D location of the center of the 2D bounding box;
• an object boundary set matching circuit to match the detected object to an existing object boundary set created from a previously received 3D image frame, the matching based on the label and the 3D location of the center of the 2D bounding box;
• and
• an object boundary set creation circuit to create, in response to a failure of the matching, a new object boundary set associated with the detected object.

At least one non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, result in the following operations for 3-Dimensional (3D) segmentation of objects, the operations comprising:

• receiving, a plurality of 3D image frames of a scene, wherein each of the 3D image frames is associated with a pose of a depth camera that generated the 3D image frames;
• detecting an object in each of the 3D image frames based on object recognition;
• associating a label with the detected object, the label derived from the object recognition;
• calculating a 2-Dimensional (2D) bounding box containing the detected object, and a 3D location of the center of the 2D bounding box;
• matching the detected object to an existing object boundary set created from a previously received 3D image frame, the matching based on the label and the 3D location of the center of the 2D bounding box;
• and
• creating in response to a failure of the matching, a new object boundary set associated with the detected object.

The computer readable storage medium of 19, further comprising:

• detecting surface planes in the scene;
• calculating an intersection of the detected surface plane with the object boundary set;
• calculating a ratio of pixels in the intersection to pixels in the detected surface plane;
• and
• if the ratio is less than a threshold value, removing the pixels in the detected surface plane from the object boundary set.