Multimodal localization and mapping for a mobile automation apparatus

A method in a mobile automation apparatus, the method comprising:

• acquiring images in a ceiling facing direction from the mobile automation apparatus;
• acquiring sensor data from a sensor configured to monitor movement of the mobile automation apparatus;
• in a mapping mode of the mobile automation apparatus:
• interfacing with a remote control to manually navigate the mobile automation apparatus and generating map data by associating the images with sensor data received from the sensor in response to the movement of the mobile automation apparatus, and
• in a localization mode of the mobile automation apparatus:
• autonomously navigating the mobile automation apparatus based on the map data generated in the mapping mode.

A device comprising:

• a communication interface;
• a memory;
• and a mapping controller configured to:
• receive, from one or more of the memory and the communication interface, images with associated sensor data, the images representing a ceiling of an environment acquired by a ceiling-facing imaging device at a mobile automation apparatus and the associated sensor data indicative of movement of the mobile automation apparatus through the environment;
• extract features from the images;
• determine, from the associated sensor data, a respective position of a mobile automation apparatus in the environment where each of the images was acquired;
• and,
• store, in the memory, extracted features from the images with associated positions.

A method comprising:

• at a device comprising:
• a communication interface;
• a memory;
• and a mapping controller, receiving, from one or more of the memory and the communication interface, images with associated sensor data, the images representing a ceiling of an environment acquired by a ceiling-facing imaging device at a mobile automation apparatus and the associated sensor data indicative of movement of the mobile automation apparatus through the environment;
• extracting, using the mapping controller, features from the images;
• determining, using the mapping controller, from the associated sensor data, a respective position of a mobile automation apparatus in the environment where each of the images was acquired;
• and,
• storing, using the mapping controller, in the memory, extracted features from the images with associated positions.