System and method of semantic segmentation for a cleaning device

A computer-implemented method for using semantic segmentation for identifying low-height obstacle avoidance of a semi-autonomous cleaning apparatus, the method comprising the steps of:

• receiving an RGB image and a depth image from the cleaning apparatus and aligning and synchronizing the RGB image and the depth image;
• generating a height image from the aligned depth image, each pixel of the height image indicating height relative to the floor;
• running a trained neural network inference using at least the height image and the RGB image to obtain a binary classification image distinguishing floor from non-floor;
• creating an obstacle point cloud from non-floor pixels in the binary classification image while excluding pixels occluded by a depth camera occlusion mask;
• converting the obstacle point cloud to a laserscan and sending the laserscan to a cost map used by a planner to generate a path that avoids the obstacles;
• and
• passing the planned path to the cleaning apparatus and executing the path to avoid the obstacles.