# US 2021/0174580 A1 · Systems and Methods for Modeling Structures Using Point Clouds Derived from Stereoscopic Image Pairs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174580 A1
- Document ID: US20210174580A1
- Status: Published application
- Kind: A1
- Application number: 16/703,644
- Filed: 2019-12-04
- Published: 2021-06-10
- Earliest claimed: 2019-12-04
- Assignees: Geomni, Inc.
- Applicants: Geomni, Inc.
- Inventors: Joseph L. Mundy; Bryce Zachary Porter; Ryan Mark Justus; Francisco Rivas
- Web page: https://patentgrep.com/patent/US20210174580A1

## CPC

- G06T 17/05: Geographic models
- G06F 16/587: using geographical or spatial information, e.g. location
- G06T 7/593: from stereo images
- G06T 15/04: Texture mapping
- G06T 15/06: Ray-tracing
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10012: Stereo images
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30184: Infrastructure
- G06T 2210/21: Collision detection, intersection
- G06T 2210/56: Particle system, point based geometry or rendering
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system for modeling a roof structure comprising an aerial imagery database and a processor in communication with the aerial imagery database. The aerial imagery database stores a plurality of stereoscopic image pairs and the processor selects at least one stereoscopic image pair among the plurality of stereoscopic image pairs and related metadata from the aerial imagery database based on a geospatial region of interest. The processor identifies a target image and a reference image from the at least one stereoscopic pair and calculates a disparity value for each pixel of the identified target image to generate a disparity map. The processor generates a three dimensional point cloud based on the disparity map, the identified target image and the identified reference image. The processor optionally generates a texture map indicative of a three-dimensional representation of the roof structure based on the generated three dimensional point cloud.

## Claims

### Claim 1

A system for modeling a structure, comprising:
an aerial imagery database; and
a processor in communication with the aerial imagery database, the processor:
retrieving at least one stereoscopic image pair from the aerial imagery database;
determining whether a three-dimensional point cloud corresponding to the at least one stereoscopic image pair exists;
if the three-dimensional point cloud corresponding to the at least one stereoscopic image pair exists, generating a three dimensional architectural polygonal model of the structure using the three-dimensional point cloud; and
if the three-dimensional point cloud corresponding to the at least one stereoscopic image pair does not exist:
identifying a target image from the at least one stereoscopic image pair;
identifying a reference image from the at least one stereoscopic image pair;
calculating a disparity value for each pixel of the identified target image;
generating a disparity map based on the calculated disparity values for each pixel of the identified target image;
generating the three dimensional point cloud based on the disparity map, the identified target image, and the identified reference image; and
generating a three dimensional architectural polygonal model of the structure using the three-dimensional point cloud.

### Claim 2 (depends on 1)

The system of claim 1, wherein the at least one stereoscopic image pair is identified using a geospatial region of interest.

### Claim 3 (depends on 1)

The system of claim 1, wherein the at least one stereoscopic image pair includes data relating to intrinsic and extrinsic camera parameters.

### Claim 4 (depends on 1)

The system of claim 1, wherein the processor calculates the disparity value for each pixel of the identified target image by applying a semi-global matching algorithm to determine a cost disparity value of each pixel of the identified target image.

### Claim 5 (depends on 1)

The system of claim 1, wherein the processor generates the three dimensional point cloud based on the disparity map and ray intersection between the identified target image and the identified reference image, the processor calculating a plurality of three dimensional intersections between respective rays passing through respective pixels in the identified target image and respective rays passing through respective corresponding pixels in the identified reference image.

### Claim 6 (depends on 1)

The system of claim 1, wherein the processor generates at least one three dimensional tool for modeling a feature over the three-dimensional point cloud.

### Claim 7 (depends on 1)

The system of claim 1, wherein the processor:
retrieves a second stereoscopic image pair from the aerial imagery database;
identifies a second target image from the second stereoscopic image pair;
identifies a second reference image from the second stereoscopic pair;
calculates a disparity value for each pixel of the identified second target image;
generates a second disparity map based on the respective calculated disparity values for each pixel of the identified second target image;
generates a second three dimensional point cloud based on the second disparity map, the identified second target image and the identified second reference image; and
generates a fused three dimensional point cloud by fusing the generated first three dimensional point cloud and the generated second three dimensional point cloud.

### Claim 8 (depends on 1)

The system of claim 1, wherein the system generates a first point cloud from a first pair of stereoscopic images, generates a second point cloud from a second pair of stereoscopic images, and automatically selects one of the first point cloud or the second point cloud for display to a user.

### Claim 9 (depends on 1)

The system of claim 1, wherein the processor executes a surface reconstruction algorithm on the three dimensional point cloud to generate a three dimensional mesh model based on the three dimensional point cloud.

### Claim 10 (depends on 1)

The system of claim 1, wherein the processor causes a graphical user interface to be displayed to an operator, the graphical user interface including the three dimensional point cloud and a user interface tool for creating a three dimensional wireframe model of a feature on top of the three-dimensional point cloud.

### Claim 11 (depends on 10)

The system of claim 10, wherein the processor receives input from the operator via the user interface tool, generates the three dimensional wireframe model of the feature based on the input from the operator, and causes the three dimensional wireframe model of the feature to be displayed on top of the three dimensional point cloud.

### Claim 12 (depends on 10)

The system of claim 10, wherein the processor applies texture from an image to at least one surface of the three dimensional wireframe model.

### Claim 13 (depends on 9)

The system of claim 9, wherein the processor generates serializable data based on the three dimensional wireframe model of the roof.

### Claim 14 (depends on 1)

The system of claim 1, wherein the processor generates a report including measurements of a real-world structure corresponding to the three-dimensional model.

### Claim 15 (depends on 14)

The system of claim 14, wherein the report is in the form of a digital file.

### Claim 16 (depends on 14)

The system of claim 14, wherein the report includes information relating to one or more of pricing information, materials, equipment, or supporting events corresponding to at least part of the three-dimensional model.

### Claim 17 (depends on 1)

The system of claim 1, wherein the processor applies a texture map to at least one surface of the three dimensional point cloud.

### Claim 18 (depends on 1)

The system of claim 1, wherein the processor colorizes points of the three dimensional point cloud.

### Claim 19

A method for modeling a structure, comprising the steps of:
receiving at least one stereoscopic image pair;
determining whether a three-dimensional point cloud corresponding to the at least one stereoscopic image pair exists;
if the three-dimensional point cloud corresponding to the at least one stereoscopic image pair exists, generating a three dimensional architectural polygonal model of the structure using the three-dimensional point cloud; and
if the three-dimensional point cloud corresponding to the at least one stereoscopic image pair does not exist:
identifying a target image from the at least one stereoscopic image pair;
identifying a reference image from the at least one stereoscopic image pair;
calculating a disparity value for each pixel of the identified target image;
generating a disparity map based on the respective calculated disparity values for each pixel of the identified target image;
generating the three dimensional point cloud based on the disparity map, the identified target image and the identified reference image; and
generating a three dimensional architectural polygonal model of the structure based on the three dimensional point cloud.

### Claim 20 (depends on 19)

The method of claim 19, further comprising identifying the at least one stereoscopic image pair using a geospatial region of interest.

### Claim 21 (depends on 19)

The method of claim 19, further comprising calculating the disparity value for each pixel of the identified target image by applying a semi-global matching algorithm to determine a cost disparity value of each pixel of the identified target image.

### Claim 22 (depends on 19)

The method of claim 19, further comprising generating the three dimensional point cloud based on the disparity map and ray intersection between the identified target image and the identified reference image, and calculating a plurality of three dimensional intersections between respective rays passing through respective pixels in the identified target image and respective rays passing through respective corresponding pixels in the identified reference image.

### Claim 23 (depends on 19)

The method of claim 19, further comprising:
receiving a second stereoscopic image pair from the aerial imagery database;
identifying a second target image from the second stereoscopic image pair;
identifying a second reference image from the second stereoscopic pair;
calculating a disparity value for each pixel of the identified second target image;
generating a second disparity map based on the respective calculated disparity values for each pixel of the identified second target image;
generating a second three dimensional point cloud based on the second disparity map, the identified second target image and the identified second reference image; and
generating a fused three dimensional point cloud by fusing the generated first three dimensional point cloud and the generated second three dimensional point cloud.

### Claim 24 (depends on 19)

The method of claim 19, further comprising executing a surface reconstruction algorithm on the three dimensional point cloud to generate a three dimensional mesh model based on the three dimensional point cloud.

### Claim 25 (depends on 19)

The method of claim 19, further comprising displaying a graphical user interface to an operator, the graphical user interface including the three dimensional point cloud and a user interface tool for creating a three dimensional wireframe model of a feature on top of the three-dimensional point cloud.

### Claim 26 (depends on 25)

The method of claim 25, further comprising receiving input from the operator via the user interface tool, generating the three dimensional wireframe model of the feature based on the input from the operator, and causing the three dimensional wireframe model of the feature to be displayed on top of the three dimensional point cloud.

### Claim 27 (depends on 25)

The method of claim 25, further comprising applying texture from an image to a surface of the three dimensional wireframe model.

### Claim 28 (depends on 20)

The method of claim 20, further comprising generating serializable data based on the three dimensional wireframe model of the roof.

### Claim 29 (depends on 19)

The method of claim 19, further comprising generating a report including measurements of a real-world structure corresponding to the three-dimensional model.

### Claim 30 (depends on 29)

The method of claim 29, wherein the report is in the form of a digital file.

### Claim 31 (depends on 29)

The method of claim 29, wherein the report includes information relating to one or more of pricing information, materials, equipment, or supporting events corresponding to at least part of the three-dimensional model.

### Claim 32 (depends on 19)

The method of claim 19, further comprising applying a texture map to at least one surface of the three dimensional point cloud.

### Claim 33 (depends on 19)

The method of claim 19, further comprising colorizing points of the three dimensional point cloud.
