Systems and methods for dynamic occlusion handling

A system for dynamic occlusion handling, the system comprising:

• a depth sensor to provide a depth map;
• a camera to provide a color image;
• and
• a processing system including at least one processing unit, the processing system being configured to implement a method that includes:
• receiving the depth map with a first boundary of an object;
• receiving the color image that corresponds to the depth map, the color image including a second boundary of the object;
• extracting depth edge points of the first boundary from the depth map;
• identifying candidates for the depth edge points along two-dimensional (2D) normal line segments in which each 2D normal line segment is perpendicular to a respective depth edge point using neighboring depth edge points;
• identifying target depth edge points on the depth map from among the corresponding candidates by determining an optimal solution for the depth edge points via discrete energy minimization with respect to an energy function that includes at least a data term based on image gradients of the color image such that positions of the target depth edge points are based on positions of corresponding color edge points of the second boundary of the object in the color image;
• and
• snapping the depth edge points to the target depth edge points such that the depth map is enhanced with an object boundary for the object.

A computer-implemented method comprising:

• receiving a depth map with a first boundary of an object;
• receiving a color image that corresponds to the depth map, the color image including a second boundary of the object;
• extracting depth edge points of the first boundary from the depth map;
• identifying candidates for the depth edge points along two-dimensional (2D) normal line segments in which each 2D normal line segment is perpendicular to a respective depth edge point using neighboring depth edge points;
• identifying target depth edge points on the depth map from among the corresponding candidates by determining an optimal solution for the depth edge points via discrete energy minimization with respect to an energy function that includes at least a data term based on image gradients of the color image such that positions of the target depth edge points are based on positions of corresponding color edge points of the second boundary of the object in the color image;
• and
• snapping the depth edge points to the target depth edge points such that the depth map is enhanced with an object boundary for the object.