Method and system for real-time geo referencing stabilization

A computing system for georeferencing stabilization, comprising:

• one or more processors;
• and
• memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
• obtaining, from one or more image capturing devices, a video stream capturing an area, wherein the video stream comprises a plurality of frames, each of the plurality of frames comprises a field of view of the image capturing device and metadata of the image capturing device when the frame is captured;
• constructing a geographic lattice for the field of view in each of the plurality of frames, wherein the geo lattice comprises a plurality of lattice points, each lattice point being associated with a pair of raw coordinates determined based on the corresponding metadata of the one or more image capturing devices;
• receiving a request for geographic coordinates of a ground point in the area;
• identifying multiple lattice map points, present in at least a first frame and a second frame of the plurality of frames, wherein the first frame and the second frame are nonconsecutive frames separated by an intervening frame, in the lattice map surrounding the ground point;
• based on a first transformation matrix between the first frame and an intervening frame, and a second transformation matrix between the intervening frame and the second frame, computing stabilized geographic coordinates of the multiple lattice map points;
• and
• computing the geographic coordinates of the ground point by interpolating the multiple pairs of stabilized geographic coordinates associated with the multiple lattice map points in the lattice map.

A non-transitory computer-readable storage medium configured with instructions executable by one or more processors to cause the one or more processors to perform operations comprising:

• obtaining, from one or more image capturing devices, a video stream capturing an area, wherein the video stream comprises a plurality of frames, each of the plurality of frames comprises a field of view of the image capturing device and metadata of the image capturing device when the frame is captured;
• constructing a geographic lattice for the field of view in each of the plurality of frames, wherein the geo lattice comprises a plurality of lattice points, each lattice point being associated with a pair of raw coordinates determined based on the corresponding metadata of the one or more image capturing devices;
• receiving a request for geographic coordinates of a ground point in the area;
• identifying multiple lattice map points, present in at least a first frame and a second frame of the plurality of frames, wherein the first frame and the second frame are nonconsecutive frames separated by an intervening frame, in the lattice map surrounding the ground point;
• based on a first transformation matrix between the first frame and an intervening frame, and a second transformation matrix between the intervening frame and the second frame, computing stabilized geographic coordinates of the multiple lattice map points;
• and
• computing the geographic coordinates of the ground point by interpolating the multiple pairs of stabilized geographic coordinates associated with the multiple lattice map points in the lattice map.