# US 8,279,218 B1 · Selecting and verifying textures in image-based three-dimensional modeling, and applications thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,279,218 B1
- Document ID: US8279218B1
- Status: Granted
- Kind: B1
- Application number: 13/245,479
- Filed: 2011-09-26
- Granted: 2012-10-02
- Earliest claimed: 2010-10-12
- Assignees: Google Inc.
- Inventors: Zhe Fan; Brian Gammon Brown; Matthew Robert Simpson; Tilman Reinhardt
- Web page: https://patentgrep.com/patent/US8279218B1

## CPC

- G06T 7/75: {involving models}
- G06T 7/593: from stereo images
- G06T 15/04: Texture mapping
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/20101: Interactive definition of point of interest, landmark or seed
- G06T 2207/30184: Infrastructure
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Embodiments relate to selecting and verifying textures in image-based three-dimensional modeling. In a first embodiment, a computer-implemented method for texturing a three-dimensional model. In the method, a polygon orientation of a polygon defining a face of a three-dimensional model is determined. A first camera orientation corresponding to a camera that took a first photographic image in a set of photographic images is also determined. The first photographic image is unconstrained to the three-dimensional model. The polygon orientation is compared with the first camera orientation to determine whether the first photographic image is directed toward the face of the three-dimensional model. If the first photographic image is directed toward the face of the three-dimensional model, a notification is provided for display to a user indicating that the first photographic image could be texture mapped to the three-dimensional model if the first photographic image is constrained to the three-dimensional model.

## Claims

### Claim 1

A computer-implemented method for selecting a texture for a face of a three-dimensional model, comprising:
(a) determining a polygon orientation of a polygon defining a face of a three-dimensional model;
(b) determining a first camera orientation corresponding to a camera that took a first photographic image in a set of photographic images, the first photographic image unconstrained to the three-dimensional model such that a user has not indicated that a position on the first photographic image corresponds to a position on the three-dimensional model;
(c) comparing the polygon orientation with the first camera orientation to determine whether the first photographic image is directed toward the face of the three-dimensional model; and
(d) if the first photographic image is directed toward the face of the three-dimensional model, providing a notification for display to a user, the notification indicating that the first photographic image could be texture mapped to the three-dimensional model if the first photographic image is constrained to the three-dimensional model.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
(e) receiving a constraint, inputted by the user, mapping a position on the first photographic image to a position on the three-dimensional model;
(f) modifying the polygon of the three-dimensional model based on the constraint; and
(g) texture mapping the first photographic image to the polygon.

### Claim 3 (depends on 2)

The method of claim 2, wherein the modifying (f) comprises applying a photogrammetry algorithm to determine a geometry of the three-dimensional model.

### Claim 4 (depends on 1)

The method of claim 1, wherein the determining (a) comprises determining a polygon vector normal to the polygon defining the face of the three-dimensional model,
wherein the determining (b) comprises determining a view vector representing an orientation of the camera when the camera took the first photographic image, and
wherein the comparing (c) comprises determining a result of a dot product of the polygon vector and the view vector, and
wherein the providing (d) comprises providing the notification when the result of the dot product exceeds a value.

### Claim 5 (depends on 4)

The method of claim 4, wherein the value is approximately 0.2.

### Claim 6 (depends on 1)

The method of claim 1, wherein further comprising:
(e) determining a second camera orientation corresponding to a camera that took a second photographic image in the set of photographic images, the second photographic image being constrained to the three-dimensional model such that a position of the second photographic image is mapped to a position on the three-dimensional model;
(f) comparing the polygon orientation with the second camera orientation to determine whether the second photographic image is directed toward the face of the three-dimensional model,
wherein the comparing (c) and notifying (d) occurs when the second photographic image is not directed toward the face of the three-dimensional model.

### Claim 7 (depends on 6)

The method of claim 6, wherein further comprising:
(g) repeating steps (a)-(f) for each face of the three-dimensional model.

### Claim 8 (depends on 1)

The method of claim 1, wherein the comparing (c) comprises determining whether the first photographic image is sufficiently directed toward the face of the three-dimensional model such that the first photographic image would not appear pixelated when texture mapped to the face of the three-dimensional model, and
wherein providing (d) occurs if the photographic image is sufficiently directed toward the face of the three-dimensional model as determined in (c).

### Claim 9 (depends on 1)

The method of claim 1, wherein the three-dimensional model comprises a three-dimensional model of a building.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
(e) determining a geometry of the three-dimensional model based on a plurality of user-inputted constraints, each constraint mapping a position on a photographic image in the set of photographic images to a position on the three-dimensional model, wherein the geometry specifies a position, orientation, and size of the face of the three-dimensional model.

### Claim 11 (depends on 1)

The method of claim 1, wherein further comprising:
(e) determining a second camera orientation corresponding to a camera that took a second photographic image in the set of photographic images, the second photographic image unconstrained to the three-dimensional model;
(f) comparing the polygon orientation with the second camera orientation to determine whether the second photographic image is directed toward the face of the three-dimensional model; and
wherein the providing (d) comprises, if the first and second photographic images are directed toward the face of the three-dimensional model, providing a notification for display to the user indicating that the face of the three-dimensional model could be texture mapped if either the first or second photographic images are constrained to the three-dimensional model.

### Claim 12

A system for selecting a texture for a face of a three-dimensional model, comprising:
an orientation module that determines a polygon orientation of a polygon defining a face of a three-dimensional model and determines a first camera orientation corresponding to a camera that took a first photographic image in a set of photographic images, the first photographic image unconstrained to the three-dimensional model such that a user has not indicated that a position on the first photographic image corresponds to a position on the three-dimensional model; and
an orientation comparison module that compares the polygon orientation with the first camera orientation to determine whether the first photographic image is directed toward the face of the three-dimensional model and, if the first photographic image is directed toward the face of the three-dimensional model, provides a notification for display to a user, the notification indicating that the first photographic image could be texture mapped to the three-dimensional model if the first photographic image is constrained to the three-dimensional model.

### Claim 13 (depends on 12)

The system of claim 12, further comprising:
a user constraint module that receives a constraint, inputted by the user, mapping a position on the photographic image to a position on the three-dimensional model;
a photogrammetry module that modifies the polygon of the three-dimensional model based on the constraint; and
a texture map module that texture maps the first photographic image to the polygon.

### Claim 14 (depends on 13)

The system of claim 13, wherein the photogrammetry module applies a photogrammetry algorithm to determine a geometry of the three-dimensional model.

### Claim 15 (depends on 12)

The system of claim 12, wherein the orientation module determines a polygon vector normal to the polygon defining the face of the three-dimensional model, and determines determining a view vector representing orientation of the camera when the camera took the photographic image, and
wherein the orientation comparison module determines a result of a dot product of the polygon vector and the view vector, and provides the notification when the result of the dot product exceeds a value.

### Claim 16 (depends on 15)

The system of claim 15, wherein the value is approximately 0.2.

### Claim 17 (depends on 12)

The system of claim 12, wherein the orientation module determines a second camera orientation corresponding to a camera that took a second photographic image in the set of photographic images, the constrained second photographic image being constrained to the three-dimensional model such that having a position of the second photographic image is mapped to a position on the three-dimensional model, and
wherein the orientation comparison module compares the polygon orientation with the second camera orientation to determine whether the second photographic image is directed toward the face of the three-dimensional model,
wherein the orientation comparison module compares the first camera orientation and the polygon orientation and provides the notification when the second photographic image is not directed toward the face of the three-dimensional model.

### Claim 18 (depends on 17)

The system of claim 17, wherein further comprising:
a GIS server that instructs the orientation module and the orientation comparison module to repeating for each face of the three-dimensional model.

### Claim 19 (depends on 12)

The system of claim 12, wherein the orientation comparison module provides the notification if the first photographic image is sufficiently directed toward the face of the three-dimensional model such that the first photographic image would not appear pixelated when texture mapped to the face of the three-dimensional model.

### Claim 20 (depends on 12)

The system of claim 12, wherein the three-dimensional model comprises a three-dimensional model of a building.

### Claim 21 (depends on 12)

The system of claim 12, further comprising:
a photogrammetry module that determining a geometry of the three-dimensional model based on a plurality of user-inputted constraints, each constraint mapping a position on a photographic image in the set of photographic images to a position on the three-dimensional model, wherein the geometry specifies a position, orientation, and size of the face of the three-dimensional model.

### Claim 22 (depends on 12)

The system of claim 12, wherein the orientation module determines a second camera orientation corresponding to a camera that took a second photographic image in the set of photographic images, the second photographic image unconstrained to the three-dimensional model; and
wherein the orientation comparison module compares the polygon orientation with the second camera orientation to determine whether the second photographic image is directed toward the face of the three-dimensional model, and, if the first and second photographic images are directed toward the face of the three-dimensional model, provides a notification for display to the user indicates that the face of the three-dimensional model could be texture mapped if either the first or second photographic images are constrained to the three-dimensional model.
