Fusion of visual-inertial-odometry and object tracker for physically anchored augmented reality

A system for presenting augmented reality content, the system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor to perform:

• receiving a video stream, comprising a plurality of frames, the video stream generated using a camera that is moving about a scene, wherein the scene includes a physical object to which the augmented reality content is to be anchored;
• determining a Visual-Inertial-Odometry camera pose for each frame in the video stream in a Visual-Inertial-Odometry coordinate system using visual inertial odometry;
• determining an Object Tracking camera pose for at least some of the frames of the video stream in an Object Tracking coordinate system using object tracking;
• determining a relation between the Visual-Inertial-Odometry coordinate system and the Object Tracking coordinate system using those frames for which both a Visual-Inertial-Odometry camera pose and an Object Tracking camera pose have been determined;
• and
• presenting augmented reality content that is placed in the video stream of the scene at a location based on the determined relation between the Visual-Inertial-Odometry coordinate system and the Object Tracking coordinate system and anchored to the physical object.

A computer program product for presenting augmented reality content, the computer program product comprising a non-transitory computer readable storage having program instructions embodied therewith, the program instructions executable by a computer, to cause the computer to perform a method comprising:

• receiving a video stream, comprising a plurality of frames, the video stream generated using a camera that is moving about a scene;
• determining a Visual-Inertial-Odometry camera pose for each frame in the video stream in a Visual-Inertial-Odometry coordinate system using visual inertial odometry;
• determining an Object Tracking camera pose for at least some of the frames of the video stream in an Object Tracking coordinate system using object tracking;
• determining a relation between the Visual-Inertial-Odometry coordinate system and the Object Tracking coordinate system using those frames for which both a Visual-Inertial-Odometry camera pose and an Object Tracking camera pose have been determined;
• and
• generating augmented reality content that is placed in the video stream of the scene at a location based on the determined relation between the Visual-Inertial-Odometry coordinate system and the Object Tracking coordinate system.