Learning an Ego State Model Through Perceptual Boosting

A robotic device comprising:

• at least one sensor;
• a control system configured to:
• capture, by the at least one sensor on the robotic device, at least one image representative of an environment;
• determine, based on the at least one image representative of the environment, a segmentation map, wherein the segmentation map segments the at least one image into a plurality of pixel areas with corresponding semantic classifications, wherein the plurality of pixel areas includes at least one pixel area with a semantic classification of a ground surface in the environment;
• adjust an ego state estimation model running on the robotic device based on the semantic classification of the ground surface in the environment, wherein the ego state estimation model is configured to maintain a pose of the robotic device;
• and
• cause the robotic device to navigate in the environment based on the adjusted ego state estimation model.

A non-transitory computer readable medium comprising program instructions executable by at least one processor to cause the at least one processor to perform functions comprising:

• capturing, by at least one sensor on a robotic device, at least one image representative of an environment;
• determining, based on the at least one image representative of the environment, a segmentation map, wherein the segmentation map segments the at least one image into a plurality of pixel areas with corresponding semantic classifications, wherein the plurality of pixel areas includes at least one pixel area with a semantic classification of a ground surface in the environment;
• adjusting an ego state estimation model running on the robotic device based on the semantic classification of the ground surface in the environment, wherein the ego state estimation model is configured to maintain a pose of the robotic device;
• and
• causing the robotic device to navigate in the environment based on the adjusted ego state estimation model.