Systems and methods to accommodate state transitions in mapping
AssigneeBaidu USA LLC
InventorsYingze BAO, Mingyu CHEN
Life of this patent
Estimating the term…
Abstract
Described herein are systems and methods that involve abnormality detection and a carefully designed state machine that assesses whether mapping, such as simultaneous localization and mapping (SLAM) processing, should be skipped for the current image frames, whether relocalization may performed, or whether SLAM processing may be performed. Thus, embodiments allow mapping processing to timely and smoothly switch between different tracking states, and thereby prevent bad tracking status to occur.
Claims21
A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by one or more processors, causes steps for performing localization and mapping of a mobile system comprising:
- initializing a number of bad image frames and a number of good image frames to starting values;
- receiving image data comprising at least one image frame, motion-related data, or both from one or more sensors of the mobile system;
- using the motion data, image data, or both to estimate a motion change of the mobile system;
- responsive to the motion change not exceeding the rapid motion threshold:
- detecting features in an image frame from the least one image frame;
- extracting feature descriptors for each of the detected features;
- responsive to the detected features not being deficient according to at least one criterion comprising number of detected features or distribution of detected features in an image frame:
- increasing the number of good image frames;
- responsive to the number of good image frames being greater than a first threshold value:
- responsive to the number of bad image frames being greater than a second threshold value, performing tracking of the mobile system;
- and
- responsive to the number of bad frames not being greater than a second threshold value, performing relocalization of the mobile system.
A processor-implement method for performing localization and mapping of a mobile system, the method comprising:
- initializing a number of bad image frames and a number of good image frames to starting values;
- receiving image data comprising at least one image frame, motion-related data, or both from one or more sensors of the mobile system;
- using the motion data, image data, or both to estimate a motion change of the mobile system;
- responsive to the motion change not exceeding the rapid motion threshold:
- detecting features in an image frame from the least one image frame;
- extracting feature descriptors for each of the detected features;
- responsive to the detected features not being deficient according to at least one criterion comprising number of detected features or distribution of detected features in an image frame:
- increasing the number of good image frames;
- responsive to the number of good image frames being greater than a first threshold value:
- responsive to the number of bad image frames being greater than a second threshold value, performing tracking of the mobile system;
- and
- responsive to the number of bad frames not being greater than a second threshold value, performing relocalization of the mobile system.
A mobile system for performing localization and mapping of the mobile system comprising:
- a vision system for capturing one or more image frames;
- one or more sensors for capturing motion-related data;
- one or more processors;
- and
- a non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by at least one processor of the one or more processors, causes steps to be performed comprising:
- initializing a number of bad image frames and a number of good image frames to starting values;
- receiving image data comprising at least one image frame, motion-related data, or both from one or more sensors of the mobile system;
- using the motion data, image data, or both to estimate a motion change of the mobile system;
- responsive to the motion change not exceeding the rapid motion threshold:
- detecting features in an image frame from the least one image frame;
- extracting feature descriptors for each of the detected features;
- responsive to the detected features not being deficient according to at least one criterion comprising number of detected features or distribution of detected features in an image frame:
- increasing the number of good image frames;
- responsive to the number of good image frames being greater than a first threshold value:
- responsive to the number of bad image frames being greater than a second threshold value, performing tracking of the mobile system;
- and
- responsive to the number of bad frames not being greater than a second threshold value, performing relocalization of the mobile system.
Dependencies are inferred from the claim text.