Mobile robot control apparatus for three dimensional modeling, three dimensional modeling system having the same and method of three dimensional modeling using the same

A three dimensional (“3D”) modeling system comprises:

• a mobile robot configured to acquire an image and generate an image sequence;
• an online 3D modeler configured to receive the image sequence from the mobile robot and to generate a first map and a second map different from the first map based on the image sequence;
• and
• a path planner configured to generate a global path based on the first map, to extract a target surface based on the second map and to generate a local inspection path having a movement unit smaller than a movement unit of the global path based on the global path and the target surface,
• wherein the path planner is configured to output a planned path to the mobile robot, and
• wherein the mobile robot is configured to displace along the planned path,
• wherein the online 3D modeler comprises:
• a pose determiner configured to receive the image sequence and to determine a pose of a reference image;
• and
• a depth estimator configured to estimate a depth of the reference image based on the reference image, the pose of the reference image, source images adjacent to the reference image and poses of the source images,
• wherein the online 3D modeler further comprises:
• a volumetric mapper configured to generate a volumetric map for determining whether an obstacle exists on a trajectory of the mobile robot or not based on the reference image, the pose of the reference image and the depth of the reference image;
• and
• a dense surfel mapper configured to generate a surfel map, which is a result of 3D modeling, based on the reference image, the pose of the reference image and the depth of the reference image,
• wherein the depth estimator is configured to output the reference image, the pose of the reference image and the depth of the reference image to both of the volumetric mapper and the dense surfel mapper,
• wherein the volumetric mapper is configured to generate the volumetric map based on the reference image, the pose of the reference image and the depth of the reference image received from the depth estimator, and
• wherein the dense surfel mapper is configured to generate the surfel map based on the reference image, the pose of the reference image and the depth of the reference image received from the depth estimator.

A method of three dimensional (“3D”) modeling, the method comprising:

• acquiring an image to generate an image sequence;
• generating a first map and a second map different from the first map based on the image sequence using an online 3D modeler;
• generating a global path based on the first map;
• extracting a target surface based on the second map;
• and
• generating a local inspection path having a movement unit smaller than a movement unit of the global path based on the global path and the target surface,
• wherein the online 3D modeler comprises:
• a pose determiner configured to receive the image sequence and to determine a pose of a reference image;
• and
• a depth estimator configured to estimate a depth of the reference image based on the reference image, the pose of the reference image, source images adjacent to the reference image and poses of the source images,
• wherein the online 3D modeler further comprises:
• a volumetric mapper configured to generate a volumetric map for determining whether an obstacle exists on a trajectory of the mobile robot or not based on the reference image, the pose of the reference image and the depth of the reference image;
• and
• a dense surfel mapper configured to generate a surfel map, which is a result of 3D modeling, based on the reference image, the pose of the reference image and the depth of the reference image,
• wherein the depth estimator is configured to output the reference image, the pose of the reference image and the depth of the reference image to both of the volumetric mapper and the dense surfel mapper,
• wherein the volumetric mapper is configured to generate the volumetric map based on the reference image, the pose of the reference image and the depth of the reference image received from the depth estimator, and
• wherein the dense surfel mapper is configured to generate the surfel map based on the reference image, the pose of the reference image and the depth of the reference image received from the depth estimator.