Estimating camera pose

Computer-implemented method of estimating a current camera pose corresponding to a current point in time using a previous camera pose corresponding to a previous point in time, of a camera configured to generate a sequence of image frames, the method comprising:

• receiving, by one or more processors, one or more image frames from the sequence of image frames, including a current image frame corresponding to the current point in time;
• receiving, by the one or more processors, inertial measurement unit data representing a motion of the camera between the previous point in time and the current point in time;
• receiving, by the one or more processors, the previous camera pose;
• inputting, by the one or more processors, the one or more image frames to one or more neural networks, and generating, using the one or more neural networks, a neural network pose prediction for the current image frame;
• adjusting, by the one or more processors, the previous camera pose using the inertial measurement unit data representing a motion of the camera between the previous point in time and the current point in time, to provide an inertial measurement unit pose prediction for the current point in time;
• and
• combining, by the one or more processors, the inertial measurement unit pose prediction for the current point in time, and the neural network pose prediction for the current image frame, to estimate the current camera pose.

A non-transitory computer-readable storage medium encoded with instructions executable by one or more processors for estimating a current camera pose corresponding to a current point in time using a previous camera pose corresponding to a previous point in time, of a camera configured to generate a sequence of image frames, the computer-readable storage medium comprising instructions to:

• receive, by one or more processors, one or more image frames from the sequence of image frames, including a current image frame corresponding to the current point in time;
• receive, by the one or more processors, inertial measurement unit data representing a motion of the camera between the previous point in time and the current point in time;
• receive, by the one or more processors, the previous camera pose;
• input, by the one or more processors, the one or more image frames to one or more neural networks, and generating, using the one or more neural networks, a neural network pose prediction for the current image frame;
• adjust, by the one or more processors, the previous camera pose using the inertial measurement unit data representing a motion of the camera between the previous point in time and the current point in time, to provide an inertial measurement unit pose prediction for the current point in time;
• and
• combine, by the one or more processors, the inertial measurement unit pose prediction for the current point in time, and the neural network pose prediction for the current image frame, to estimate the current camera pose.