Camera tracking via dynamic perspectives

A computer product comprising a non-transitory computer readable medium storing a plurality of instructions that when executed control a computer system to perform:

• identifying first positions of a first physical camera and a second physical camera, wherein the first positions correspond to a first time period;
• rendering, using an animation engine, a first virtual scene for the first time period;
• projecting the first virtual scene onto a display surface to determine a first rendered image for the first time period, the projecting of the first virtual scene consistent with the first positions of the first physical camera and the second physical camera;
• storing the first rendered image in a frame buffer for displaying on the display surface;
• receiving a first camera image of the display surface, the first camera image obtained using the second physical camera during the first time period;
• measuring a movement of the first physical camera during the first time period;
• determining a first corrected position of the second physical camera based at least in part on the movement of the first physical camera;
• predicting, using the movement of the first physical camera and the first corrected position, second positions of the first physical camera and the second physical camera, wherein the second positions correspond to a second time period in which a second rendered image is to be displayed on the display surface;
• rendering, using the animation engine, a second virtual scene for the second time period;
• and
• projecting the second virtual scene onto the display surface to determine the second rendered image for the second time period, the projecting of the second virtual scene consistent with the second positions of the first physical camera and the second physical camera.

A system comprising:

• a first physical camera and a second physical camera;
• a non-transitory computer readable medium;
• and
• one or more processors communicably coupled with the non-transitory computer readable medium and configured to perform:
• identifying first positions of the first physical camera and the second physical camera, wherein the first positions correspond to a first time period;
• rendering, using an animation engine, a first virtual scene for the first time period;
• projecting the first virtual scene onto a display surface to determine a first rendered image for the first time period, the projecting of the first virtual scene consistent with the first positions of the first physical camera and the second physical camera;
• storing the first rendered image in a frame buffer for displaying on the display surface;
• receiving a first camera image of the display surface, the first camera image obtained using the second physical camera during the first time period;
• measuring a movement of the first physical camera during the first time period;
• determining a first corrected position of the second physical camera based at least in part on the movement of the first physical camera;
• predicting, using the movement of the first physical camera and the first corrected position, second positions of the first physical camera and the second physical camera, wherein the second positions correspond to a second time period in which a second rendered image is to be displayed on the display surface;
• rendering, using the animation engine, a second virtual scene for the second time period;
• and
• projecting the second virtual scene onto the display surface to determine the second rendered image for the second time period, the projecting of the second virtual scene consistent with the second positions of the first physical camera and the second physical camera.