Methods, apparatus, and systems for localization and mapping

A mobile device, comprising:

• a dead reckoning sensor configured to obtain motion data during a movement of the mobile device;
• a visual sensor configured to acquire a plurality of images of an environment in which the mobile device operates;
• and
• a controller operably coupled with the dead reckoning sensor and the visual sensor, and configured to:
• acquire an image through the visual sensor during the movement of the mobile device;
• match the image with one or more key frames stored in a key frame database, wherein the one or more key frames are created based on two-dimensional coordinates of feature points included in the plurality of images previously acquired by the visual sensor;
• compute a visual relative pose based on two-dimensional coordinates of matching feature points included in both of the image and the one or more key frames that have been matched with the image;
• compute relevant information of the visual relative pose based on the two-dimensional coordinates of the matching feature points;
• and
• update an absolute pose of the mobile device and a map based on the relevant information of the visual relative pose and relevant information of a dead reckoning based relative pose obtained based on the motion data.

A non-transitory computer-readable storage medium storing computer-executable instructions, which when executed by a processor, are configured to cause the processor to perform a method comprising:

• acquiring an image through a visual sensor during a movement of a mobile device;
• matching the image with one or more key frames stored in a key frame database, wherein the one or more key frames are created based on two-dimensional coordinates of feature points included in a plurality of images previously acquired through the visual sensor;
• computing a visual relative pose based on two-dimensional coordinates of matching feature points included in both of the image and the one or more key frames that have been matched with the image;
• computing relevant information of the visual relative pose based on the two-dimensional coordinates of the matching feature points;
• and
• updating an absolute pose of the mobile device and a map based on the relevant information of the visual relative pose and relevant information of a dead reckoning based relative pose.