# US 2020/0404167 A1 · Automated guide for image capturing for 3d model creation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0404167 A1
- Document ID: US20200404167A1
- Status: Published application
- Kind: A1
- Application number: 17/010,625
- Filed: 2020-09-02
- Published: 2020-12-24
- Earliest claimed: 2017-01-11
- Assignees: Hover Inc.
- Applicants: Hover Inc.
- Inventors: Manish Upendran; William Castillo; Derek Halliday
- Web page: https://patentgrep.com/patent/US20200404167A1

## CPC

- G06T 7/50: Depth or shape recovery
- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- H04N 23/633: {for displaying additional information relating to control or operation of the camera}
- H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet
- H04N 23/71: Circuitry for evaluating the brightness variation
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2200/28: involving image processing hardware
- G06T 2207/30168: Image quality inspection
- G06T 2207/30184: Infrastructure
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image capture system provides automated prompts for aiding a user in capturing images for use in 3D model creation. While a user is preparing to capture an image, the system provides visual indications that indicate whether a quality-based condition is satisfied. Based on the visual indications, a user can determine whether an image, if captured, would likely be suitable for use in creating a 3D model. Determining if the quality-based condition is satisfied may include monitoring output generated by one or more sensors and comparing the output against a threshold value. Additionally, the system may analyze the visual content or metadata associated with an image to determine if the quality-based condition is satisfied and request user input to further identify certain image features that were identified by the system.

## Claims

### Claim 1

A method for capturing images used to create 3D models, comprising:
displaying, on a device, visual content comprising a target object that would be captured as an image if a capture control for the device were to be activated;
determining, by one or more processors, whether a quality-based condition is satisfied;
wherein determining whether the quality-based condition is satisfied comprises determining, by one or more processors, whether the entirety of the target object is visible in the visual content;
at least while the quality-based condition is not satisfied, displaying, on the device, a visual indication associated with the quality-based condition concurrent with displaying the visual content;
capturing the visual content displayed on the device, as an image, in response to activation of the capture control.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
detecting that the quality-based condition is satisfied; and
in response to detecting that the quality-based condition is satisfied, ceasing to display the visual indication.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
detecting that the quality-based condition is satisfied; and
in response to detecting that the quality-based condition is satisfied, changing an appearance of the visual indication.

### Claim 4 (depends on 1)

The method of claim 1, wherein:
the target object is a building; and
determining, by one or more processors, whether the entirety of the target object is visible in the visual content comprises determining whether the visual content comprises one or more of: building corners, a top of the building, or a bottom of the building.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining whether the entirety of the target object is visible in the visual content comprises:
generating a bounding box around the target object; and
determining whether the entire bounding box is within a frame of the visual content.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining whether the entirety of the target object is visible in the visual content comprises performing edge detection for the target object.

### Claim 7 (depends on 6)

The method of claim 6, wherein performing edge detection for the target object comprises:
determining a type of the target object;
identifying a set of edges associated with the type of the target object; and
determining whether all of the identified set of edges are visible in the visual content.

### Claim 8 (depends on 7)

The method of claim 7, wherein:
the type of the target object is a building type;
the set of edges comprise one or more outer building edges; and
determining whether all of the identified set of edges are visible in the visual content comprises determining whether all of the one or more outer building edges are visible in the visual content.

### Claim 9 (depends on 1)

The method of claim 1, wherein the quality-based condition indicates whether the visual content displayed on the device is suitable for use in creating a 3D model of the target object.

### Claim 10

One or more non-transitory computer-readable media storing one or more sequences of instructions that, when executed by one or more processors, cause:
displaying, on a device, visual content comprising a target object that would be captured as an image if a capture control for the device were to be activated;
determining, by one or more processors, whether a quality-based condition is satisfied;
wherein determining whether the quality-based condition is satisfied comprises determining, by one or more processors, whether the entirety of the target object is visible in the visual content;
at least while the quality-based condition is not satisfied, displaying, on the device, a visual indication associated with the quality-based condition concurrent with displaying the visual content;
capturing the visual content displayed on the device, as an image, in response to activation of the capture control.

### Claim 11 (depends on 10)

The one or more non-transitory computer-readable media of claim 10, wherein the one or more sequences of instructions further comprise instructions that, when executed by one or more processors, cause:
detecting that the quality-based condition is satisfied; and
in response to detecting that the quality-based condition is satisfied, ceasing to display the visual indication.

### Claim 12 (depends on 10)

The one or more non-transitory computer-readable media of claim 10, wherein the one or more sequences of instructions further comprise instructions that, when executed by one or more processors, cause:
detecting that the quality-based condition is satisfied; and
in response to detecting that the quality-based condition is satisfied, changing an appearance of the visual indication.

### Claim 13 (depends on 10)

The one or more non-transitory computer-readable media of claim 10, wherein:
the target object is a building; and
determining, by one or more processors, whether the entirety of the target object is visible in the visual content comprises determining whether the visual content comprises one or more of: building corners, a top of the building, or a bottom of the building.

### Claim 14 (depends on 10)

The one or more non-transitory computer-readable media of claim 10, wherein determining whether the entirety of the target object is visible in the visual content comprises:
generating a bounding box around the target object; and
determining whether the entire bounding box is within a frame of the visual content.

### Claim 15 (depends on 10)

The one or more non-transitory computer-readable media of claim 10, wherein determining whether the entirety of the target object is visible in the visual content comprises performing edge detection for the target object.

### Claim 16 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein performing edge detection for the target object comprises:
determining a type of the target object;
identifying a set of edges associated with the type of the target object; and
determining whether all of the identified set of edges are visible in the visual content.

### Claim 17 (depends on 16)

The one or more non-transitory computer-readable media of claim 16, wherein:
the type of the target object is a building type;
the set of edges comprise one or more outer building edges; and
determining whether all of the identified set of edges are visible in the visual content comprises determining whether all of the one or more outer building edges are visible in the visual content.

### Claim 18 (depends on 10)

The one or more non-transitory computer-readable media of claim 10, wherein the quality-based condition indicates whether the visual content displayed on the device is suitable for use in creating a 3D model of the target object.
