US 12,602,823 B2Granted

Re-localization of robot

AssigneeIntel Corporation

InventorsXuesong Shi, Yuxin Tian, Sangeeta Ghangam, Dawei Wang

Life of this patent

Estimating the term…

Loading patent…

Abstract

A method for re-localization of the robot may include retrieving, for each of keyframes in a keyframe database of the robot, image features and a pose of the keyframe, the image features of the keyframe comprising a global descriptor and local descriptors of the keyframe (210); extracting image features of a current frame captured by the robot, the image features of the current frame comprising a global descriptor and local descriptors of the current frame (220); determining one or more rough matching frames from the keyframes based on comparison between the global descriptor of each keyframe and the global descriptor of the current frame (230); determining a final matching frame from the one or more rough matching frames based on comparison between the local descriptors of each rough matching frame and the local descriptors of the current frame (240); and calculating a pose of the current frame based on a pose of the current frame based on a pose of the final matching frame (250).

Claims20

Dependent claims

A method for re-localization of a robot, comprising:

retrieving, for each keyframe of a set of keyframes in a keyframe database of the robot, image features and a pose of the keyframe, the image features of the keyframe comprising a global descriptor and local descriptors of the keyframe;
extracting image features of a current frame captured by the robot, the image features of the current frame comprising a global descriptor and local descriptors of the current frame;
determining one or more rough matching frames from the set of keyframes based on comparison between the global descriptor of each keyframe and the global descriptor of the current frame;
determining a final matching frame from the one or more rough matching frames based on comparison between the local descriptors of each rough matching frame and the local descriptors of the current frame, wherein determining the final matching frame comprises:
for each rough matching frame,
determining, for each local descriptor of the current frame, a matching point in the rough matching frame corresponding to a keypoint in the current frame represented by the local descriptor of the current frame;
calculating a fundamental matrix or a homography matrix between the current frame and the rough matching frame based on the keypoint in the current frame and the matching point in the rough matching frame;
calculating a reprojection error between the keypoint in the current frame and the matching point in the rough matching frame based on the fundamental matrix or the homography matrix;
and
determining the matching point as an inline point when the reprojection error is smaller than a predetermined threshold, and
selecting a rough matching frame with a largest number of inline points as the final matching frame;
and
calculating a pose of the current frame based on a pose of the final matching frame.

10 more elements

An apparatus for re-localization of a robot, comprising interface circuitry;

processor circuitry coupled to the interface circuitry and configured to:
retrieve, for each keyframe of a set of keyframes in a keyframe database of the robot, image features and a pose of the keyframe from the keyframe database via the interface circuitry, the image features of the keyframe comprising a global descriptor and local descriptors of the keyframe;
extract image features of a current frame captured by the robot, the image features of the current frame comprising a global descriptor and local descriptors of the current frame;
determine one or more rough matching frames based on comparison between the global descriptor of each keyframe and the global descriptor of the current frame;
determine a final matching frame based on comparison between the local descriptors of each rough matching frame and the local descriptors of the current frame, wherein the processor circuitry configured to determine the final matching frame comprises the processor circuitry to:
for each rough matching frame,
determine, for each local descriptor of the current frame, a matching point in the rough matching frame corresponding to a keypoint in the current frame represented by the local descriptor of the current frame;
calculate a fundamental matrix or a homography matrix between the current frame and the rough matching frame based on the keypoint in the current frame and the matching point in the rough matching frame;
calculate a reprojection error between the keypoint in the current frame and the matching point in the rough matching frame based on the fundamental matrix or the homography matrix;
and
determine the matching point as an inline point when the reprojection error is smaller than a predetermined threshold, and
select a rough matching frame with a largest number of inline points as the final matching frame;
and
calculate a pose of the current frame based on a pose of the final matching frame.

11 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/254,181Filed 25 Dec 2020
Published: US 2024/0029300 A125 Jan 2024
Granted: 14 Apr 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification16 codes

G01C 21/20
Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00: instruments for performing navigational calculations
G06T 7/74
Using feature-based methods: involving reference images or patches
G05D 1/0274
Using internal positioning means: using mapping information stored in a memory device
G06T 1/0014
General purpose image data processing: image feed-back for automatic industrial control, e.g. robot with camera