Pose tracking for rolling shutter camera

A non-transitory computer-readable medium with instructions stored thereon, wherein the medium is configured to be incorporated in an augmented reality system configured to track poses of a rolling shutter camera, that when executed by a processor, perform the steps comprising:

• capturing an image, the image comprising a plurality of image lines;
• estimating a reference pose of the camera at a reference line, the reference line being one of the plurality of the image lines;
• estimating relative poses of the camera to the reference pose of the camera using inertial sensor data, each relative pose of the camera corresponding to an image line other than the reference line;
• refining the reference pose of the camera based on the relative poses of the camera in combination with information contained in the image.

An eyewear device, wherein the eyewear is configured to be incorporated in an augmented reality system comprising:

• a rolling shutter camera;
• an inertial sensor;
• wherein the poses of the camera are configured to be tracked with steps comprising:
• capturing an image, the image comprising a plurality of image lines;
• estimating a reference pose of the camera at a reference line, the reference line being one of the plurality of the image lines;
• estimating relative poses of the camera to the reference pose of the camera using inertial sensor data, each relative pose of the camera corresponding to an image line other than the reference line;
• refining the reference pose of the camera based on the relative poses of the camera in combination with information contained in the image.