Graphical coordinate system transform for video frames

A computing method, comprising:

• computing feature points in a new frame and a prior frame of a series of successive video frames;
• computing optical flow vectors between the prior frame and new frame;
• determining a homography transform between the new frame and prior frame based upon the computed feature points and the computed optical flow vectors for each frame;
• applying the homography transform to the graphical element coordinate system in the prior frame to thereby generate an updated graphical element coordinate system in the new frame;
• generating a six degree of freedom camera pose transform between the graphical element coordinate system in the prior frame and the updated graphical element coordinate system in the new frame based on the homography transform and a camera pose of the graphical element coordinate system in the prior frame;
• and
• rendering an updated graphical element in the new frame relative to the updated graphical element coordinate system using the six degree of freedom camera pose transform.

A computing device, comprising:

• a processor configured to analyze a series of successive video frames, and further configured to:
• in an initialization phase:
• initialize a data structure defining video frame types including an initial frame, a new frame, a last matched frame, the current reference frame, and prior reference frames;
• receive an initial video frame of the series of video frames;
• receive user input parameters including a camera parameter, a quad defined in a graphical element coordinate system, and a graphical element coordinate system rotation and translation;
• compute a projection of the quad to the initial video frame;
• compute feature points of the initial video frame;
• store a bitmap of the initial video frame and the computed feature points as a current reference frame;
• in a matching phase, for each of a plurality of successive frames in the series:
• receive the successive frame as a new frame;
• compute feature points for the new frame;
• select a candidate frame to match with the new frame, the candidate frame selected from the group consisting of the last matched frame, the current reference frame, and one or more of the prior reference frames, the selection being based on whether an area of interest including the projected quad is visible in the last matched frame, and further based on a size of a candidate transform computed to match the new frame to the reference frame, the candidate frame including a projected quad;
• compute a sparse optical flow between an area of interest including the projected quad in the candidate frame and a corresponding area of interest in the new frame;
• create a set of features including vectors computed by the sparse optical flow computation and point pairs computed in the computation of feature points;
• calculate a plurality homography transform candidates using the optical flow vectors and feature points of the set of features, using a modified RANSAC algorithm with all point pairs and vectors in the set of features, and a truncated set of features as input for the RANSAC algorithm, the truncated set of features having inlier point pairs and inlier vectors removed therefrom;
• rank the plurality of homography transform candidates based upon factors including number of inlier point pairs and inlier vectors and distance to the area of interest, a difference metric between a bitmap of the reference frame as transformed by the candidate transform and a bitmap of the new frame;
• select a top ranked homography transform;
• apply the top ranked homography transform to the projected quad in the candidate frame to thereby generate an updated quad in the new frame;
• and
• generate a six degree of freedom camera pose transform based on the homography transform, a camera pose of the quad in the prior frame, and the user input parameters.