# US 2019/0378328 A1 · Generating multi-dimensional building models with ground level images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378328 A1
- Document ID: US20190378328A1
- Status: Published application
- Kind: A1
- Application number: 16/544,327
- Filed: 2019-08-19
- Published: 2019-12-12
- Earliest claimed: 2013-07-23
- Assignees: Hover Inc.
- Applicants: Hover Inc.
- Inventors: Shaohui Sun; Ioannis Pavlidis; Adam J. Altman
- Web page: https://patentgrep.com/patent/US20190378328A1

## CPC

- G06T 17/05: Geographic models
- G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units}
- G06T 15/04: Texture mapping
- G06T 15/20: Perspective computation
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 19/003: {Navigation within 3D models or images}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06V 20/176: {Urban or other man-made structures}
- G06V 20/653: {by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces}
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2210/04: Architectural design, interior design
- G06T 2210/56: Particle system, point based geometry or rendering
- G06T 2219/2008: Assembling, disassembling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system and method is provided for generating textured building models from wirelessly captured ground-level imagery. Ground-level images of façades of building objects are collected for identification of geometry planes and generation of building façade geometry. The 3D building model is properly geo-positioned, scaled and textured.

## Claims

### Claim 1

A method of modeling planar surfaces, the method comprises:
receiving a plurality of ground-level images from a smartphone, the plurality of ground-level images corresponding to all sides of an entire building;
creating a point cloud with data points from the received plurality of ground-level images;
selecting a plurality of vertices defining edges of at least one single planar surface within the point cloud;
outlining the at least one single planar surface based on the selected plurality of vertices; and
creating a simplified façade geometry for the at least one single planar surface based on the outlined single planar surface.

### Claim 2 (depends on 1)

The method of claim 1, wherein creating a simplified geometry further comprises correlating data points within the point cloud to the outlined single planar surface.

### Claim 3 (depends on 2)

The method of claim 2 further comprises:
receiving an orthogonal image for the entire building;
identifying a plurality of orthogonal vertices of the orthogonal image;
correlating at least two of the plurality of orthogonal vertices with at least one single planar surface; and
constructing a building model representing all surfaces of the entire building using at least the correlated plurality of orthogonal vertices.

### Claim 4 (depends on 2)

The method of claim 2 further comprises averaging a planar fit for a plurality of the data points within the outlined single planar surface.

### Claim 5 (depends on 4)

The method of claim 4 further comprises adjusting a plane position of at least one single planar surface based on the average planar fit.

### Claim 6 (depends on 1)

The method of claim 1 further comprises applying a texture to the at least one planar surface.

### Claim 7 (depends on 6)

The method of claim 6, wherein the applying a texture comprises mapping at least one of the ground-level images to the at least one planar surface.

### Claim 8 (depends on 6)

The method of claim 6, wherein the applying a texture comprises using one or more of:
camera information or ground level image information.

### Claim 9

A method of modeling planar surfaces, the method comprises:
receiving a plurality of ground-level images from a smartphone, the plurality of ground-level images corresponding to a plurality of sides of a building;
creating a three-dimensional (3D) point cloud with data points from the received plurality of ground-level images;
selecting a plurality of vertices defining edges of a plurality of planar surfaces within the 3D point cloud;
outlining the plurality of planar surfaces based on the selected plurality of vertices; and
creating a simplified façade geometry for the plurality of sides of the building based on the outlined plurality of planar surfaces.

### Claim 10 (depends on 9)

The method of claim 9 further comprises averaging a planar fit for the data points within the outlined plurality of planar surfaces.

### Claim 11 (depends on 10)

The method of claim 10 further comprises adjusting a plane position of at least one the plurality of planar surfaces based on the average planar fit.

### Claim 12 (depends on 9)

The method of claim 9 further comprises applying a texture to the plurality of planar surfaces.

### Claim 13 (depends on 12)

The method of claim 12, wherein the applying a texture comprises mapping at least one of the ground-level images to a corresponding planar surface of the plurality of planar surfaces.

### Claim 14 (depends on 12)

The method of claim 12, wherein the applying a texture comprises using one or more of: camera information or ground-level image information.

### Claim 15 (depends on 9)

The method of claim 9 further comprises:
receiving an orthogonal image for an entire perimeter of the building;
identifying a plurality of orthogonal vertices of the orthogonal image;
correlating at least two of the plurality of orthogonal vertices with each of the plurality of planar surfaces; and
constructing a building model representing all surfaces of the building using at least the correlated plurality of orthogonal vertices.

### Claim 16

A system for modeling planar surfaces, the system comprises:
a computer system for receiving, from a wireless capture device, a plurality of ground-level images of a building object captured by the wireless capture device circumnavigating a building object;
an image database for storing the received plurality of ground-level images of a building object;
an image processor, operative with the image database, to:
create a point cloud with data points from the received ground-level images;
select a plurality of vertices defining edges of at least one single planar surface within the point cloud;
outline the at least one single planar surface based on the selected plurality of vertices; and
create a simplified façade geometry for the at least one single planar surface based on the outlined single planar surface.

### Claim 17 (depends on 16)

The system of claim 16, wherein creating a simplified geometry further comprises correlating data points within the point cloud to the outlined single planar surface.

### Claim 18 (depends on 17)

The system of claim 17 further comprises:
receiving an orthogonal image for the entire building;
identifying a plurality of orthogonal vertices of the orthogonal image;
correlating at least two of the plurality of orthogonal vertices for each of the at least one single planar surfaces; and
constructing a building model representing all surfaces of the entire building using at least the correlated plurality of orthogonal vertices.

### Claim 19 (depends on 17)

The system of claim 17 further comprises averaging a planar fit for the data points within the outlined single planar surface.

### Claim 20 (depends on 19)

The system of claim 19 further comprises adjusting a plane position of at least one single planar surface based on the average planar fit.

### Claim 21 (depends on 16)

The system of claim 16 further comprises applying a texture to the at least one planar surface.

### Claim 22 (depends on 21)

The system of claim 21, wherein the applying a texture comprises mapping at least one of the ground-level images to the at least one planar surface.

### Claim 23 (depends on 21)

The system of claim 21, wherein the applying a texture comprises using one or more of:
camera information or ground level image information.

### Claim 24 (depends on 16)

The system of claim 16 further comprises a three dimensional (3D) proximity buffer applied to selected data points in the point cloud to determine the at least one planar surface and identify data points in the point cloud that are in close proximity to a region around each of the selected data points.

### Claim 25 (depends on 24)

The system of claim 24, wherein a closest actual data point to the region is used as the data point to determine the at least one planar surface.
