Computational budget estimation for vision-aided inertial navigation systems

A vision-aided inertial navigation system comprising:

• at least one image source to produce image data along a trajectory of the vision-aided inertial navigation system (VINS) within an environment, wherein the image data contains a plurality of features observed within the environment at a plurality of poses of the VINS along the trajectory;
• an inertial measurement unit (IMU) to produce IMU data indicative of motion of the vision-aided inertial navigation system;
• and
• a hardware-based processing unit executing an estimator that determines, based on the image data and the IMU data, estimates for at least a position and orientation of the vision-aided inertial navigation system for a plurality of poses of the VINS along the trajectory,
• wherein the estimator determines the estimates by:
• computing a budget specifying an amount of resources that can be allocated to state estimations for a current pose of the VINS,
• classifying, for each of the poses and based on the budget, each of the features observed at the respective pose into either a first set of the features or a second set of the features,
• maintaining a state vector having states for a position and orientation of the VINS and for positions with the environment for the first set of features for a sliding window of two or more of the most recent poses along the trajectory without maintaining states for positions of the second set of features within the state vector, and
• computing state estimates in accordance with the selected features.