# US 2021/0264561 A1 · Automatic multi-image 3d ground control point extraction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0264561 A1
- Document ID: US20210264561A1
- Status: Published application
- Kind: A1
- Application number: 16/800,846
- Filed: 2020-02-25
- Published: 2021-08-26
- Earliest claimed: 2020-02-25
- Assignees: Raytheon Company
- Applicants: Raytheon Company
- Inventors: Richard W. Ely; Jody D. Verret
- Web page: https://patentgrep.com/patent/US20210264561A1

## CPC

- G06T 3/14: Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T 7/74: {involving reference images or patches}
- G01C 11/02: Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
- G06T 7/337: {involving reference images or patches}
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/30181: Earth observation
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)

## Abstract

Discussed herein are devices, systems, and methods for multi-image ground control point (GCP) determination. A method can include extracting, from a first image including image data of a first geographical region, a first image template, the first image template including a contiguous subset of pixels from the first image and a first pixel of the first image indicated by the GCP, predicting a first pixel location of the GCP in a second image, the second image including image data of a second geographical overlapping with the first geographical region, extracting, from the second image, a second image template, the second image template including a contiguous subset of pixels from the second image and a second pixel corresponding to the pixel location, identifying a second pixel of the second image corresponding to a highest correlation score, and adding a second pixel location of the identified pixel to the GCP.

## Claims

### Claim 1

A computer-implemented method for multi-image ground control point (GCP) determination, the method comprising:
extracting, from a first image including image data of a first geographical region, a first image template, the first image template including a contiguous subset of pixels from the first image and a first pixel of the first image indicated by the GCP;
predicting a first pixel location of the GCP in a second image, the second image including image data of a second geographical overlapping with the first geographical region;
extracting, from the second image, a second image template, the second image template including a contiguous subset of pixels from the second image and a second pixel corresponding to the pixel location;
identifying a second pixel of the second image corresponding to a highest correlation score; and
adding a second pixel location of the identified pixel to the GCP.

### Claim 2 (depends on 1)

The method of claim 1, further comprising determining respective correlation scores, using a normalized cross-correlation, for the first image template centered at a variety of pixels of the second image template.

### Claim 3 (depends on 2)

The method of claim 2, further comprising comparing a highest score of the correlation scores to a first threshold value and discarding the second pixel location if the ratio is less than the second threshold value.

### Claim 4 (depends on 3)

The method of claim 3, further comprising comparing a ratio, of the highest correlation score to a second highest correlation score, to a second threshold value and discarding the second pixel location if the ratio is less than the second threshold value.

### Claim 5 (depends on 1)

The method of claim 1, wherein the first pixel is the center of the first image template.

### Claim 6 (depends on 1)

The method of claim 1, wherein the second pixel is the center of the second image template.

### Claim 7 (depends on 1)

The method of claim 1, further comprising warping the second image template using an affine transformation before identifying the second pixel.

### Claim 8 (depends on 7)

The method of claim 7, further comprising projecting the identified second pixel to an image space of the second pixel to determine the second pixel location.

### Claim 9

A non-transitory machine-readable medium including instructions that, when executed by a machine, cause a machine to perform operations for determining a multi-image ground control point (GCP), the operations comprising:
extracting, from a first image including image data of a first geographical region, a first image template, the first image template including a contiguous subset of pixels from the first image and a first pixel of the first image indicated by the GCP;
predicting a first pixel location of the GCP in a second image, the second image including image data of a second geographical overlapping with the first geographical region;
extracting, from the second image, a second image template, the second image template including a contiguous subset of pixels from the second image and a second pixel corresponding to the pixel location;
identifying a second pixel of the second image corresponding to a highest correlation score; and
adding a second pixel location of the identified pixel to the GCP.

### Claim 10 (depends on 9)

The non-transitory machine-readable medium of claim 9, wherein the operations further comprise determining respective correlation scores, using a normalized cross-correlation, for the first image template centered at a variety of pixels of the second image template.

### Claim 11 (depends on 10)

The non-transitory machine-readable medium of claim 10, wherein the operation further comprise comparing a highest score of the correlation scores to a first threshold value and discarding the second pixel location if the ratio is less than the second threshold value.

### Claim 12 (depends on 11)

The non-transitory machine-readable medium of claim 11, wherein the operations further comprise comparing a ratio, of the highest correlation score to a second highest correlation score, to a second threshold value and discarding the second pixel location if the ratio is less than the second threshold value.

### Claim 13 (depends on 9)

The non-transitory machine-readable medium of claim 9, wherein the first pixel is the center of the first image template.

### Claim 14 (depends on 9)

The non-transitory machine-readable medium of claim 9, wherein the second pixel is the center of the second image template.

### Claim 15 (depends on 9)

The non-transitory machine-readable medium of claim 9, wherein the operations further comprise warping the second image template using an affine transformation before identifying the second pixel.

### Claim 16 (depends on 15)

The non-transitory machine-readable medium of claim 15, wherein the operations further comprise projecting the identified second pixel to an image space of the second pixel to determine the second pixel location.

### Claim 17

A system comprising:
a memory including image data of first and second images of a geographical region stored thereon;
processing circuitry coupled to the memory, the processing circuitry configured to:
extract, from the first image, a first image template, the first image template including a contiguous subset of pixels from the first image and a first pixel of the first image indicated by the GCP;
predict a first pixel location of the GCP in the second image;
extract, from the second image, a second image template, the second image template including a contiguous subset of pixels from the second image and a second pixel corresponding to the pixel location;
identify a second pixel of the second image corresponding to a highest correlation score; and
add a second pixel location of the identified pixel to the GCP.

### Claim 18 (depends on 17)

The system of claim 17, wherein the processing circuitry is further configured to determine respective correlation scores, using a normalized cross-correlation, for the first image template centered at a variety of pixels of the second image template.

### Claim 19 (depends on 17)

The system of claim 17, wherein the processing circuitry is further configured to compare a highest score of the correlation scores to a first threshold value and discard the second pixel location if the ratio is less than the second threshold value.

### Claim 20 (depends on 19)

The system of claim 19, wherein the processing circuitry is further configured to compare a ratio, of the highest correlation score to a second highest correlation score, to a second threshold value and discarding the second pixel location if the ratio is less than the second threshold value.
