Robot generating map based on multi sensors and artificial intelligence and moving based on map

A robot capable of moving using a map generated based on multi sensors and artificial intelligence, the robot comprising:

• a motor configured to move the robot;
• a light detection and ranging (LiDAR) sensor configured to determine a distance between an external object and the robot;
• a plurality of camera sensors configured to capture an image of the external object;
• a map storage configured to store a first graph including:
• a LiDAR branch including one or more LiDAR frames generated by the LiDAR sensor, each of the one or more LiDAR frames generated for a respective position of the robot, and one or more LiDAR keyframes selected from among the one or more LiDAR frames,
• a visual branch including one or more visual frames generated by the camera sensors, each of the one or more visual frames generated for a respective position of the robot, and one or more visual keyframes selected from among the one or more visual frames,
• a backbone including a plurality of frame nodes, wherein each of the frame nodes corresponds to a respective position of the robot that is same as at least a respective position corresponding to a LiDAR keyframe of the one or more LiDAR keyframes or a respective position corresponding to a visual keyframe of the one or more visual keyframes, and
• odometry information;
• and
• a controller configured to determine a current location of the robot by using the odometry information or by comparing a frame node of the frame nodes with a first LiDAR frame of the one or more LiDAR frames or a first visual frame of the one or more visual frames,
• wherein the odometry information comprises rotation and direction information of a wheel while the robot moves between a first location correlated with a first frame node from among the frame nodes and a second location correlated with a second frame node from among the frame nodes,
• wherein the controller is further configured to select the one or more visual keyframes from among the one or more visual frames based on the one or more visual keyframes having more 3D points than a first criterion, and
• wherein the controller is further configured to select the one or more LiDAR keyframes from among the one or more LiDAR frames based on an overlapped size of a selected LiDAR keyframe with other LiDAR keyframes being smaller than a second criterion.

A robot, the robot comprising:

• a motor configured to move the robot;
• a plurality of sensors;
• a map storage configured to store:
• a pose graph including
• a LiDAR branch including one or more LiDAR frames generated by the sensors, each of the one or more LiDAR frames generated for a respective position of the robot, and one or more LiDAR keyframes selected from among the one or more LiDAR frames,
• a visual branch including one or more visual frames generated by the sensors, each of the one or more visual frames generated for a respective position of the robot, and one or more visual keyframes selected from among the one or more visual frames,
• a backbone including a plurality of frame nodes, wherein each of the frame nodes corresponds to a respective position of the robot that is same as at least a respective position corresponding to a LiDAR keyframe of the one or more LiDAR keyframes or a respective position corresponding to a visual keyframe of the one or more visual keyframes, and
• odometry information between the frame nodes;
• and
• a controller configured to calculate a current position of the robot by comparing a frame registered with a frame node of the frame nodes with a LiDAR frame of the one or more LiDAR frames or a visual frame of the one or more visual frames, or by using the odometry information,
• wherein the controller is further configured to select the one or more visual keyframes from among the one or more visual frames based on the one or more visual keyframes having more 3D points than a first criterion, and
• wherein the controller is further configured to select the one or more LiDAR keyframes from among the one or more LiDAR frames based on an overlapped size of a selected LiDAR keyframe with other LiDAR keyframes being smaller than a second criterion.