# US 2023/0065774 A1 · Computer Vision Systems and Methods for Modeling Three-Dimensional Structures Using Two-Dimensional Segments Detected in Digital Aerial Images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0065774 A1
- Document ID: US20230065774A1
- Status: Published application
- Kind: A1
- Application number: 17/983,069
- Filed: 2022-11-08
- Published: 2023-03-02
- Earliest claimed: 2018-06-29
- Assignees: Insurance Services Office, Inc.
- Applicants: Insurance Services Office, Inc.
- Inventors: Jose Luis Esteban; Raul Cabido; Francisco Rivas
- Web page: https://patentgrep.com/patent/US20230065774A1

## CPC

- G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
- G06F 9/3891: {organised in groups of units sharing resources, e.g. clusters}
- G06T 7/55: from multiple images
- G06T 7/579: from motion
- G06T 15/205: {Image-based rendering}
- G06T 17/05: Geographic models
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06V 10/757: {Matching configurations of points or features}
- G06V 20/13: Satellite images
- G06V 20/176: {Urban or other man-made structures}
- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/30184: Infrastructure
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system for modeling a three-dimensional structure utilizing two-dimensional segments comprising a memory and a processor in communication with the memory. The processor extracts a plurality of two-dimensional segments corresponding to the three-dimensional structure from a plurality of images indicative of different views of the three-dimensional structure. The processor determines a plurality of three-dimensional candidate segments based on the extracted plurality of two-dimensional segments and adds the plurality of three-dimensional candidate segments to a three-dimensional segment cloud. The processor transforms the three-dimensional segment cloud into a wireframe indicative of the three-dimensional structure by performing a wireframe extraction process on the three-dimensional segment cloud.

## Claims

### Claim 1

A system for modeling a three-dimensional structure utilizing two-dimensional segments comprising:
a memory; and
a processor in communication with the memory, the processor:
extracting a plurality of two-dimensional segments corresponding to the three-dimensional structure from a plurality of images indicative of different views of the three-dimensional structure;
determining a plurality of three-dimensional candidate segments based on the extracted plurality of two-dimensional segments;
adding the plurality of three-dimensional candidate segments to a three-dimensional segment cloud; and
transforming the three-dimensional segment cloud into a wireframe indicative of the three-dimensional structure by performing a wireframe extraction process on the three-dimensional segment cloud.

### Claim 2 (depends on 1)

The system of claim 1, wherein the processor:
captures the plurality of images from different camera viewpoints;
determines a projection plane, camera parameter sets, and image parameters associated with each image of the plurality of images; and
identifies, based on the projection plane, the camera parameter sets, and the image parameters, two-dimensional segments sets in each image of the plurality of images corresponding to edges of the three-dimensional structure.

### Claim 3 (depends on 1)

The system of claim 1, wherein the processor determines the plurality of three-dimensional candidate segments based on the plurality of two-dimensional segments by determining a plurality of ground three-dimensional segments, determining a plurality of horizontal three-dimensional segments, and determining a plurality of oblique three-dimensional segments.

### Claim 4 (depends on 3)

The system of claim 3, wherein the processor determines the plurality of ground three-dimensional segments by:
pairing each extracted two-dimensional segment to proximate extracted two-dimensional segments in views other than ground views of the three-dimensional structure;
selecting parallel two-dimensional segment pairs;
determining a three-dimensional segment from each selected parallel two-dimensional segment pair; and
selecting ground three-dimensional segments from the determined three-dimensional segments.

### Claim 5 (depends on 3)

The system of claim 3, wherein the processor determines the plurality of horizontal three-dimensional segments by:
pairing each extracted two-dimensional segment with parallel extracted two-dimensional segments in views other than horizontal views of the three-dimensional structure;
determining a three-dimensional segment from each two-dimensional segment pair;
compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and
selecting at least one cluster from the plurality of clusters and determining a horizontal three-dimensional segment indicative of the selected cluster.

### Claim 6 (depends on 3)

The system of claim 3, wherein the processor determines the plurality of oblique three-dimensional segments by:
pairing each extracted two-dimensional segment with non-parallel extracted two-dimensional segments in views other than oblique views of the three-dimensional structure;
determining a three-dimensional segment from each two-dimensional segment pair;
compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and
selecting at least one cluster from the plurality of clusters and determining an oblique three-dimensional segment indicative of the selected cluster.

### Claim 7 (depends on 1)

The system of claim 1, wherein the processor determines the plurality of three-dimensional segments based on the extracted plurality of two-dimensional segments by determining a plurality of epipolar three-dimensional segments, the processor:
pairing extracted two-dimensional segments in other views with compatible epipolar lines, the epipolar lines reducing comparisons between extracted two-dimensional segments;
determining a three-dimensional segment from each two-dimensional segment pair;
selecting three-dimensional segment pairs having a consensus above a predetermined threshold value; and
excluding outlier three-dimensional segment pairs.

### Claim 8

A method for modeling a three-dimensional structure utilizing two-dimensional segments comprising the steps of:
extracting a plurality of two-dimensional segments corresponding to the three-dimensional structure from a plurality of images indicative of different views of the three-dimensional structure;
determining a plurality of three-dimensional candidate segments based on the extracted plurality of two-dimensional segments;
adding the plurality of three-dimensional candidate segments to a three-dimensional segment cloud; and
transforming the three-dimensional segment cloud into a wireframe indicative of the three-dimensional structure by performing a wireframe extraction process on the three-dimensional segment cloud.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
capturing the plurality of images from different camera viewpoints;
determining a projection plane, camera parameter sets, and image parameters associated with each image of the plurality of images; and
identifying, based on the projection plane, the camera parameter sets, and the image parameters, the two-dimensional segments sets in each image of the plurality of images corresponding to edges of the three-dimensional structure.

### Claim 10 (depends on 8)

The method of claim 8, further comprising determining a plurality of ground three-dimensional segments, determining a plurality of horizontal three-dimensional segments, and determining a plurality of oblique three-dimensional segments.

### Claim 11 (depends on 10)

The method of claim 10, further comprising determining the plurality of ground three-dimensional segments by:
pairing each extracted two-dimensional segment to proximate extracted two-dimensional segments in views other than ground views of the three-dimensional structure;
selecting parallel two-dimensional segment pairs;
determining a three-dimensional segment from each selected parallel two-dimensional segment pair; and
selecting ground three-dimensional segments from among the determined three-dimensional segments.

### Claim 12 (depends on 10)

The method of claim 10, further comprising determining the plurality of horizontal three-dimensional segments by:
pairing each extracted two-dimensional segment with parallel extracted two-dimensional segments in views other than horizontal views of the three-dimensional structure;
determining a three-dimensional segment from each two-dimensional segment pair;
compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and
selecting at least one cluster from the plurality of clusters and determining a horizontal three-dimensional segment indicative of the selected cluster.

### Claim 13 (depends on 10)

The method of claim 10, further comprising determining the plurality of oblique three-dimensional segments by:
pairing each extracted two-dimensional segment with non-parallel extracted two-dimensional segments in views other than oblique views of the three-dimensional structure;
determining a three-dimensional segment from each two-dimensional segment pair;
compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and
selecting at least one cluster from the plurality of clusters and determining an oblique three-dimensional segment indicative of the selected cluster.

### Claim 14 (depends on 8)

The method of claim 8, further comprising determining a plurality of epipolar three-dimensional segments by:
pairing extracted two-dimensional segments in other views with compatible epipolar lines, the epipolar lines reducing comparisons between extracted two-dimensional segments;
determining a three-dimensional segment from each two-dimensional segment pair;
selecting three-dimensional segment pairs having a consensus above a predetermined threshold value; and
excluding outlier three-dimensional segment pairs.

### Claim 15

A non-transitory computer readable medium having instructions stored thereon for modeling a three-dimensional structure utilizing two-dimensional segments which, when executed by a processor, causes the processor to carry out the steps of:
extracting a plurality of two-dimensional segments corresponding to the three-dimensional structure from a plurality of images indicative of different views of the three-dimensional structure;
determining a plurality of three-dimensional candidate segments based on the extracted plurality of two-dimensional segments;
adding the plurality of three-dimensional candidate segments to a three-dimensional segment cloud; and
transforming the three-dimensional segment cloud into a wireframe indicative of the three-dimensional structure by performing a wireframe extraction process on the three-dimensional segment cloud.

### Claim 16 (depends on 15)

The non-transitory computer readable medium of claim 15, the processor further carrying out the steps of:
capturing the plurality of images from different camera viewpoints;
determining a projection plane, camera parameter sets, and image parameters associated with each image of the plurality of images; and
identifying, based on the projection plane, the camera parameter sets and the image parameters, the two-dimensional segments sets in each image of the plurality of images corresponding to edges of the three-dimensional structure.

### Claim 17 (depends on 15)

The non-transitory computer readable medium of claim 15, the processor further carrying out the steps of determining a plurality of ground three-dimensional segments, determining a plurality of horizontal three-dimensional segments, and determining a plurality of oblique three-dimensional segments.

### Claim 18 (depends on 17)

The non-transitory computer readable medium of claim 17, the processor determining the plurality of ground three-dimensional segments by carrying out the steps of:
pairing each extracted two-dimensional segment with proximate extracted two-dimensional segments in views other than ground views of the three-dimensional structure;
selecting parallel two-dimensional segment pairs;
determining a three-dimensional segment from each selected parallel two-dimensional segment pair; and
selecting ground three-dimensional segments from the determined three-dimensional segments.

### Claim 19 (depends on 17)

The non-transitory computer readable medium of claim 17, the processor determining the plurality of horizontal three-dimensional segments by carrying out the steps of:
pairing each extracted two-dimensional segment with parallel extracted two-dimensional segments in views other than horizontal views of the three-dimensional structure;
determining a three-dimensional segment from each two-dimensional segment pair;
compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and
selecting at least one cluster from the plurality of clusters and determining a horizontal three-dimensional segment indicative of the selected cluster.

### Claim 20 (depends on 17)

The non-transitory computer readable medium of claim 17, the processor determining the plurality of oblique three-dimensional segments by carrying out the steps of:
pairing each extracted two-dimensional segment with non-parallel extracted two-dimensional segments in views other than oblique views of the three-dimensional structure;
determining a three-dimensional segment from each two-dimensional segment pair;
compiling the determined three-dimensional segments, based on characteristics thereof, into a plurality of clusters; and
selecting at least one cluster from the plurality of clusters and determining an oblique three-dimensional segment indicative of the selected cluster.

### Claim 21 (depends on 15)

The non-transitory computer readable medium of claim 15, the processor determining a plurality of epipolar three-dimensional segments by carrying out the steps of:
pairing extracted two-dimensional segments in other views with compatible epipolar lines, the epipolar lines reducing comparisons between extracted two-dimensional segments;
determining a three-dimensional segment from each two-dimensional segment pair;
selecting three-dimensional segment pairs having a consensus above a predetermined threshold value; and
excluding outlier three-dimensional segment pairs.
