# US 12,597,162 B2 · System calibration using remote sensor data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,597,162 B2 - Document ID: US12597162B2 - Status: Granted - Kind: B2 - Application number: 17/721,915 - Filed: 2022-04-15 - Published: 2023-10-19 - Granted: 2026-04-07 - Earliest claimed: 2022-04-15 - Assignees: Cavnue Technology, LLC - Applicants: Cavnue Technology, LLC - Inventors: Katerina Patrice Schulz; Vishal Mandal; Abhishek Baral; Cyrus Blankinship; David Hahn Clifford - Web page: https://patentgrep.com/patent/US12597162B2 ## CPC - G06T 7/248: {involving reference images or patches} - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G01S 19/485: {whereby the further system is an optical system or imaging system} - G06T 7/74: {involving reference images or patches} - G06T 11/40: Filling planar surfaces by adding surface attributes, e.g. adding colours or textures - G06V 10/245: {by locating a pattern; Special marks for positioning} - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06V 20/54: of traffic, e.g. cars on the road, trains or boats - G06T 2207/30184: Infrastructure - G06T 2207/30236: Traffic on road, railway or crossing - G06T 2207/30244: Camera pose - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Methods, systems, and apparatus, including computer programs encoded on computer storage media, for monitoring characteristics of cameras used for road surveillance. In some implementations, a plurality of first image frames are obtained from a camera monitoring a specific geographical area. For each first image frame of the plurality of first image frames, a second image frame is generated by adjusting a viewpoint of the first image frame. A third image frame is generated by rasterizing the second image frame. Photonic content is identified in the third image frame. Invariant components are determined in a plurality of third image frames based on the photonic content identified in subsequent third image frames. Position changes of the camera are determined by identifying position changes of the invariant components in the subsequent third image frames. In response to determining the position changes of the camera, a position of the camera is adjusted. ## Claims ### Claim 1 A computer-implemented method comprising: obtaining, by one or more processors, a plurality of first image frames from a camera that is monitoring a specific geographical area from an elevated viewpoint; for each first image frame of the plurality of first image frames from the camera that is monitoring the specific geographical area from the elevated viewpoint: generating, by the one or more processors, a second image frame from the first image frame from the camera that is monitoring the specific geographical area from the elevated viewpoint by generating vertical projection of the first image frame to a birds-eye view; generating, by the one or more processors, a third image frame from the second image frame that is a vertical projection of the first image frame by converting pixel values of the second image frame to produce rasterized pixel data in the third image frame; and identifying, by the one or more processors, photonic content from the rasterized pixel data in the third image frame; determining, by the one or more processors, one or more invariant components classified as stationary in a plurality of third image frames based on the photonic content identified in subsequent third image frames; determining, by the one or more processors, one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames; comparing, by the one or more processors, the one or more position changes of the camera to a threshold value; and in response to determining the one or more position changes of the camera satisfies the threshold value, adjusting, by the one or more processors, a position of the camera. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein obtaining the plurality of first image frames from the camera monitoring the specific geographic area comprises: determining, by the one or more processors, one or more characteristics of the camera based on content of the plurality of first image frames; and adjusting, by the one or more processors, the one or more characteristics of the camera for monitoring the specific geographical area. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein obtaining, by the one or more processors, a plurality of fourth image frames of the specific geographic area, wherein each fourth image frame of the plurality of fourth image frames include content of the specific geographical area and imagery of the camera; identifying, by the one or more processors, one or more geographical components in each first image frame of the plurality of first image frames using geographical content from the plurality of fourth image frames; and determining, by the one or more processors, a geographical field of view of the camera based on the one or more geographical components in each first image frame of the plurality of first image frames. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, wherein the plurality of fourth image frames are obtained from at least one of a satellite, an unmanned aerial vehicle, and an airplane. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein generating the second image frame from the first image frame by adjusting the elevated viewpoint of the first image frame comprises: transposing, by the one or more processors, the first image frame by an orthogonal projection to the second image frame, wherein the second image frame is a 90 degree rotated viewpoint of the first image frame; and assigning, by the one or more processors, data indicative of cardinality to the second image frame. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein identifying the photonic content from the rasterized pixel data in the third image frame comprises: identifying, by the one or more processors, one or more edges in the third image frame; identifying, by the one or more processors, one or more contours in the third image frame; and identifying, by the one or more processors, a specific color in the third image frame. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein determining the one or more invariant components classified as stationary in the plurality of third image frames based on the photonic content identified in subsequent third image frames comprises: obtaining, by the one or more processors, the plurality of third image frames by aggregating each subsequent third image frame; and detecting, by the one or more processors, one or more static features in each of the plurality of third image frames by (i) comparing features between subsequent third image frames, (ii) identifying the one or more static features between the subsequent third image frames of the features that do not change location, and (iii) identifying one or more non static features between the subsequent third image frames of the features that do change location. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein determining the one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames further comprises: determining, by the one or more processors, a location for each of the one or more static features in the plurality of third image frames; for each third image frame of the plurality of third image frames: determining, by the one or more processors, a difference amount between the location of each of the one or more static features in between an initial third image frame and a subsequent third image frame; comparing, by the one or more processors, the difference amount to a second threshold value; and in response to determining the difference amount satisfies the second threshold value, identifying, by the one or more processors, a position change of the camera. ### Claim 9 (depends on 7) The computer-implemented method of claim 7, wherein determining the one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames further comprises: determining, by the one or more processors, a location for each of the one or more static features in the plurality of third image frames; for each third image frame of the plurality of third image frames: determining, by the one or more processors, a difference amount between the location of each of the one or more static features in between the third image frame and an initial raster frame; comparing, by the one or more processors, the difference amount to a threshold value; and in response to determining the difference amount satisfies the threshold value, identifying, by the one or more processors, a position change of the camera from an initial calibrated position. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein adjusting the position of the camera further comprises at least one of: recalibrating, by the one or more processors, optical characteristics of the camera to match to an initial calibrated position of the camera; or adjusting, by the one or more processors, the position of the camera to the initial calibrated position of the camera. ### Claim 11 (depends on 1) The computer-implemented method of claim 1, wherein the one or more invariant components represent geographic components comprising at least one of a pole, a lane marker, a hill, a median, crosswalks, mountain, and a tree. ### Claim 12 A system comprising: one or more processors; one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: obtaining, by the one or more processors, a plurality of first image frames from a camera that is monitoring a specific geographical area from an elevated viewpoint; for each first image frame of the plurality of first image frames from the camera that is monitoring the specific geographical area from the elevated viewpoint: generating, by the one or more processors, a second image frame from the first image frame from the camera that is monitoring the specific geographical area from the elevated viewpoint by generating vertical projection of the first image frame to a birds-eye view; generating, by the one or more processors, a third image frame from the second image frame that is a vertical projection of the first image frame by converting pixel values of the second image frame to produce rasterized pixel data in the third image frame; and identifying, by the one or more processors, photonic content from the rasterized pixel data in the third image frame; determining, by the one or more processors, one or more invariant components classified as stationary in a plurality of third image frames based on the photonic content identified in subsequent third image frames; determining, by the one or more processors, one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames; comparing, by the one or more processors, the one or more position changes of the camera to a threshold value; and in response to determining the one or more position changes of the camera satisfies the threshold value, adjusting, by the one or more processors, a position of the camera. ### Claim 13 (depends on 12) The system of claim 12, wherein obtaining the plurality of first image frames from the camera monitoring the specific geographic area further comprises: determining, by the one or more processors, one or more characteristics of the camera based on content of the plurality of first image frames; and adjusting, by the one or more processors, the one or more characteristics of the camera for monitoring the specific geographical area. ### Claim 14 (depends on 12) The system of claim 12, wherein obtaining, by the one or more processors, a plurality of fourth image frames of the specific geographic area, wherein each fourth image frame of the plurality of fourth image frames include content of the specific geographical area and imagery of the camera; identifying, by the one or more processors, one or more geographical components in each first image frame of the plurality of first image frames using geographical content from the plurality of fourth image frames; and determining, by the one or more processors, a geographical field of view of the camera based on the one or more geographical components in each first image frame of the plurality of first image frames. ### Claim 15 (depends on 14) The system of claim 14, wherein the plurality of fourth image frames are obtained from at least one of a satellite, an unmanned aerial vehicle, and an airplane. ### Claim 16 (depends on 12) The system of claim 12, wherein generating the second image frame from the first image frame by adjusting the elevated viewpoint of the first image frame comprises: transposing, by the one or more processors, the first image frame by an orthogonal projection to the second image frame, wherein the second image frame is a 90 degree rotated viewpoint of the first image frame; and assigning, by the one or more processors, data indicative of cardinality to the second image frame. ### Claim 17 (depends on 16) The system of claim 16, wherein identifying the photonic content from the rasterized pixel data in the third image frame comprises: identifying, by the one or more processors, one or more edges in the third image frame; identifying, by the one or more processors, one or more contours in the third image frame; and identifying, by the one or more processors, a specific color in the third image frame. ### Claim 18 (depends on 12) The system of claim 12, wherein determining the one or more invariant components classified as stationary in the plurality of third image frames based on the photonic content identified in subsequent third image frames further comprises: obtaining, by the one or more processors, the plurality of third image frames by aggregating each subsequent third image frame; and detecting, by the one or more processors, one or more static features in each of the plurality of third image frames by (i) comparing features between subsequent third image frames, (ii) identifying the one or more static features between the subsequent third image frames of the features that do not change location, and (iii) identifying one or more non static features between the subsequent third image frames of the features that do change location. ### Claim 19 (depends on 18) The system of claim 18, wherein determining the one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames comprises: determining, by the one or more processors, a location for each of the one or more static features in the plurality of third image frames; for each third image frame of the plurality of third image frames: determining, by the one or more processors, a difference amount between the location of each of the one or more static features in between an initial third image frame and a subsequent third image frame; comparing, by the one or more processors, the difference amount to a threshold value; and in response to determining the difference amount satisfies the threshold value, identifying, by the one or more processors, a position change of the camera. ### Claim 20 A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising: obtaining, by one or more processors, a plurality of first image frames from a camera that is monitoring a specific geographical area from an elevated viewpoint; for each first image frame of the plurality of first image frames from the camera that is monitoring the specific geographical area from the elevated viewpoint: generating, by the one or more processors, a second image frame from the first image frame from the camera that is monitoring the specific geographical area from the elevated viewpoint by generating vertical projection of the first image frame to a birds-eye view; generating, by the one or more processors, a third image frame from the second image frame that is a vertical projection of the first image frame by converting pixel values of the second image frame to produce rasterized pixel data in the third image frame; and identifying, by the one or more processors, photonic content from the rasterized pixel data in the third image frame; determining, by the one or more processors, one or more invariant components classified as stationary in a plurality of third image frames based on the photonic content identified in subsequent third image frames; determining, by the one or more processors, one or more position changes of the camera by measuring differences between locations indicated by the rasterized pixel data corresponding to the one or more invariant components between successive third image frames in the plurality of third image frames; comparing, by the one or more processors, the one or more position changes of the camera to a threshold value; and in response to determining the one or more position changes of the camera satisfies the threshold value, adjusting, by the one or more processors, a position of the camera.