Systems and methods for performing self-improving visual odometry

A system comprising:

• one or more processors;
• one or more non-transitory computer-readable media including one or more sequences of instructions which, when executed by the one or more processors, causes:
• receiving, by a neural network implemented using one or more computer systems, a plurality of images of an environment;
• determining, by the neural network, for each image, a respective set of interest points and a respective descriptor;
• determining, by the neural network, a correspondence between the plurality of images based on the sets of interest points and the descriptors of the plurality of images, wherein determining the correspondence between the plurality of images comprises:
• determining one or point correspondences between the sets of interest points, wherein each point correspondence indicates a respective correspondence between an interest point of one image and an interest point of another image, and
• determining, based on the one or more point correspondences, a set of candidate interest points, each candidate interest point indicating a respective feature in the environment in three-dimensional space;
• and
• determining, for each candidate interest point, a respective stability metric;
• and modifying the neural network based on the one or more candidate interest points.