# US 2022/0284609 A1 · Image analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284609 A1
- Document ID: US20220284609A1
- Status: Published application
- Kind: A1
- Application number: 17/638,773
- Filed: 2020-08-27
- Published: 2022-09-08
- Earliest claimed: 2019-08-28
- Assignees: Hover Inc.
- Applicants: Hover Inc.
- Inventors: Atulya SHREE; Kai JIA; Zhiyao XIONG; Siu Fai CHOW; Raymond PHAN; Panfeng LI; Domenico CURRO
- Web page: https://patentgrep.com/patent/US20220284609A1

## CPC

- G06T 7/543: from line drawings
- G06T 7/579: from motion
- G06T 7/12: Edge-based segmentation
- G06T 7/33: using feature-based methods
- G06T 7/73: using feature-based methods
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Techniques are described for identifying correspondences between images to generate a fundamental matrix for the camera positions related to the images. The resultant fundamental matrix enables epipolar geometry to correlate common features among the images. Correspondences are identified by confirming feature matches across images by applying a homography to data representing features across images. Further techniques are described herein for generating a representation of a boundary of a feature of a structure based on a digital image. In one or more embodiments, generating a representation of a boundary of a particular feature in a digital image comprises determining a portion of the image that corresponds to the structure, and determining a portion of the image that corresponds to the particular feature. One more vanishing points are associated with the portion of the image corresponding to the particular feature. The one or more vanishing points are used to generate a set of bounding lines for the particular feature, based on which the boundary indicator for the feature is generated.

## Claims

### Claim at position 1

1-48. (canceled)

### Claim 49

A computer-implemented method comprising:
receiving a first image comprising data for a set of representations of co-planar rectangular features on a first surface having a first orientation;
receiving a second image comprising data for representations of the set of co-planar rectangular features on the first surface having a second orientation;
selecting, from the set of representations on the first surface having the first orientation, a first representation, wherein the set of representations includes one or more first additional representations on the first surface having the first orientation in addition to the first representation;
selecting, from the set of representations on the first surface having the second orientation, a second representation, wherein the set of representations includes one or more second additional representations on the first surface having the second orientation in addition to the second representation;
generating a first homography aligning the first representation to the second representation; and
estimating locations of the first additional representations on the first surface having the second orientation based on the first homography.

### Claim 50 (depends on 49)

The method of claim 49, further comprising generating a fundamental matrix between the first and second images based on the first homography.

### Claim 51 (depends on 50)

The method of claim 50, further comprising:
drawing an epipolar line in the second image based on the fundamental matrix with the first image; and
correlating at least one feature in the first image in the second image.

### Claim 52 (depends on 51)

The method of claim 51, further comprising estimating at least one camera position based on the correlated features.

### Claim 53 (depends on 52)

The method of claim 52, further comprising reconstructing a three dimensional model of the correlated features based on the estimated camera position.

### Claim 54 (depends on 49)

The method of claim 49, further comprising validating the first homography based on a comparison between the estimated locations of one or more of the first additional representations relative to the second additional representations.

### Claim 55 (depends on 54)

The method of claim 54, wherein validating further comprises detecting non-alignment for all estimated locations of the first additional representations and at least one of the second additional representations.

### Claim 56 (depends on 54)

The method of claim 54, wherein validating further comprises detecting an error below a threshold for a position of the estimated locations of at least one of the first additional representations and at least one of the second additional representations.

### Claim 57 (depends on 56)

The method of claim 56, further comprising adjusting the first homography based on the first representation and the data representing the at least one first additional representation.

### Claim 58 (depends on 57)

The method of claim 57, further comprising generating a fundamental matrix between the first and second images based on the adjusted first homography.

### Claim 59 (depends on 54)

The method of claim 54, wherein validating further comprises detecting that a particular feature representation, of the first additional representations, is an inlier based on determining that the estimated location of the particular feature representation is within a threshold number of pixels from locations of at least one of the second additional representations.

### Claim 60 (depends on 59)

The method of claim 59, further comprising:
determining that one or more other feature representations, other than the particular feature representation of the first additional representations, are mis-aligned; and
in response to determining that the one or more other feature representations of the first additional representations, are mis-aligned, adjusting the first homography.

### Claim 61 (depends on 60)

The method of claim 60, further comprising generating a fundamental matrix between the first and second images based on the adjusted first homography.

### Claim 62

A computer-readable storage medium comprising instructions which, when executed by a computer system, cause the computer system to:
receive a first image comprising data for a set of representations of co-planar rectangular features on a first surface having a first orientation;
receive a second image comprising data for representations of the set of co-planar rectangular features on the first surface having a second orientation;
select, from the set of representations on the first surface having the first orientation, a first representation, wherein the set of representations includes one or more first additional representations on the first surface having the first orientation in addition to the first representation;
select, from the set of representations on the first surface having the second orientation, a second representation, wherein the set of representations includes one or more second additional representations on the first surface having the second orientation in addition to the second representation;
generate a first homography aligning the first representation to the second representation; and
estimate locations of the first additional representations on the first surface having the second orientation based on the first homography.

### Claim 63 (depends on 62)

The computer-readable storage medium of claim 62, further comprising instructions that, when executed by a computer system, cause the computer system to generate a fundamental matrix between the first and second images based on the first homography.

### Claim 64 (depends on 63)

The computer-readable storage medium of claim 63, further comprising instructions that, when executed by a computer system, cause the computer system to:
draw an epipolar line in the second image based on the fundamental matrix with the first image; and
correlate at least one feature in the first image in the second image.

### Claim 65 (depends on 64)

The computer-readable storage medium of claim 64, further comprising instructions that, when executed by a computer system, cause the computer system to estimate at least one camera position based on the correlated features.

### Claim 66 (depends on 65)

The computer-readable storage medium of claim 65, further comprising instructions that, when executed by a computer system, cause the computer system to reconstruct a three dimensional model of the correlated features based on the estimated camera position.

### Claim 67 (depends on 62)

The computer-readable storage medium of claim 62, further comprising instructions that, when executed by a computer system, cause the computer system to validate the first homography based on a comparison between the estimated locations of one or more of the first additional representations relative to the second additional representations.

### Claim 68 (depends on 67)

The computer-readable storage medium of claim 67, wherein the instructions, when executed by a computer system, cause the computer system to validate the first homography by detecting non-alignment for all estimated locations of the first additional representations and at least one of the second additional representations.

### Claim 69 (depends on 67)

The computer-readable storage medium of claim 67, wherein the instructions, when executed by a computer system, cause the computer system to validate the first homography by detecting an error below a threshold for a position of the estimated locations of at least one of the first additional representations and at least one of the second additional representations.

### Claim 70 (depends on 69)

The computer-readable storage medium of claim 69, further comprising instructions that, when executed by a computer system, cause the computer system to:
adjust the first homography based on the first representation and the data representing the at least one first additional representation.

### Claim 71 (depends on 70)

The computer-readable storage medium of claim 70, further comprising instructions that, when executed by a computer system, cause the computer system to generate a fundamental matrix between the first and second images based on the adjusted first homography.

### Claim 72 (depends on 67)

The computer-readable storage medium of claim 67, wherein the instructions, when executed by the computer system, cause the computer system to validate the first homography by detecting that a particular feature representation, of the first additional representations, is an inlier based on determining that the estimated location of the particular feature representation is within a threshold number of pixels from locations of at least one of the second additional representations.

### Claim 73 (depends on 72)

The computer-readable storage medium of claim 72, further comprising instructions that, when executed by a computer system, cause the computer system to:
determine that one or more other feature representations, other than the particular feature representation of the first additional representations, are mis-aligned; and
in response to determining that the one or more other feature representations of the first additional representations, are mis-aligned, adjust the first homography.

### Claim 74 (depends on 73)

The computer-readable storage medium of claim 73, further comprising instructions that, when executed by a computer system, cause the computer system to generate a fundamental matrix between the first and second images based on the adjusted first homography.