# US 2025/0166326 A1 · Systems and methods for generating dimensionally coherent training data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0166326 A1
- Document ID: US20250166326A1
- Status: Published application
- Kind: A1
- Application number: 18/841,339
- Filed: 2023-02-23
- Published: 2025-05-22
- Earliest claimed: 2022-02-23
- Assignees: Hover Inc.
- Applicants: Hover, Inc.
- Inventors: Zhiyao XIONG
- Web page: https://patentgrep.com/patent/US20250166326A1

## CPC

- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 7/74: {involving reference images or patches}
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 15/04: Texture mapping
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06N 20/00: Machine learning
- G06T 2207/20081: Training; Learning
- G06T 2207/30244: Camera pose
- G06T 2210/04: Architectural design, interior design
- G06T 2210/12: Bounding box
- G06T 2210/44: Morphing
- G06T 2215/12: Shadow map, environment map
- G06T 2219/2016: Rotation, translation, scaling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

System and method are provided for generating training data for feature matching among images of a building structure. The method includes obtaining a model of a building that includes a camera solution and images used to generate the geometric model. The method also includes, for facades of the model: applying a minimum bounding box to a respective facade to obtain a respective facade slice that is a 2-D plane represented in a 3-D coordinate system of the model; and projecting visual data of at least one camera in the camera solution that viewed the respective facade onto a visibility mask associated with the respective facade slice. The method also includes photo-texturing the projected visual data facade slice to one of the facade slices or the geometric model to generate a visual 3-D representation of the building; and generating a training dataset by perturbing the visual 3-D representation.

## Claims

### Claim 1

A method for generating training data for feature matching among images of a building structure, the method comprising:
obtaining a geometric model of a building that includes a camera solution and a plurality of images used to generate the geometric model;
for a plurality of façades of the geometric model:
applying a minimum bounding box to a respective façade to obtain a respective façade slice that is a 2-D plane represented in a 3-D coordinate system of the geometric model; and
projecting visual data of at least one camera in the camera solution that viewed the respective façade onto a visibility mask associated with the respective façade slice;
photo-texturing the projected visual data to one of the respective façade slice or the geometric model to generate a visual 3-D representation of the building; and
generating a training dataset by perturbing the visual 3-D representation.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining cameras that viewed the respective façade by reprojecting boundaries of the minimum bounding box into one or more images of the plurality of images.

### Claim 3 (depends on 1)

The method of claim 1, wherein a camera view of a respective façade comprises one or more occlusions of the respective façade, the method further comprising:
generating cumulative texture information for the respective façade based on visual information of one or more additional cameras of the camera solution.

### Claim 4 (depends on 3)

The method of claim 3, wherein generating the cumulative texture information comprises:
projecting a visibility mask for the respective façade within a segmentation mask in an image of a set of partially occluded images that shows the respective façade.

### Claim 5 (depends on 1)

The method of claim 1, wherein projecting visual data of cameras that viewed the respective façade comprises:
for each camera that viewed the respective façade:
transforming image data of the respective façade to generate a respective morphed image; and
merging visual data from each morphed image to the respective façade slice.

### Claim 6 (depends on 5)

The method of claim 5, wherein transforming the image data of the respective façade uses a homogenous transformation.

### Claim 7 (depends on 5)

The method of claim 5, wherein generating the respective morphed image orients a plane of the respective façade orthogonal to an optical axis of a virtual camera viewing the transformed image data.

### Claim 8 (depends on 5)

The method of claim 5, wherein merging the visual data comprises:
selecting a base template from amongst visibility masks for the respective façade, wherein the base template has a largest observed pixel volume relative to other visibility masks;
applying visual data from an image of a set of partially occluded images that shows the respective façade; and
importing pixels from other images to unobserved areas of the base template.

### Claim 9 (depends on 1)

The method of claim 1, wherein the perturbing is performed relative to one or more cameras of the camera solution at each perturbation performed on the visual 3-D representation in that position.

### Claim 10 (depends on 1)

The method of claim 1, wherein the perturbing is performed relative to one or more virtual cameras viewing the visual 3-D representation.

### Claim 11 (depends on 1)

The method of claim 1, wherein the perturbing includes one or more of: moving, rotating, zooming, and combinations thereof.

### Claim 12 (depends on 1)

The method of claim 1, further comprising capturing new images of the visual 3-D representation from each perturbed position.

### Claim 13 (depends on 1)

The method of claim 1, further comprising:
performing feature matching by identifying matched features across images of the training dataset for transforming camera positions between images and camera poses.

### Claim 14

A method for generating training data for feature matching among images of a building structure, the method comprising:
obtaining a geometric model of a building that includes a camera solution and a plurality of images used to generate the geometric model;
for each facade of a plurality of façades of the geometric model, generating a respective façade slice;
identifying a respective camera within the camera solution that observes each façade slice;
for each identified camera, identifying pixels comprising visual data associated with a respective façade slice according to a visibility mask for the respective façade slice within the respective identified camera's associated image;
generating an aggregate façade view of cumulative visual data for the respective façade slice from each identified camera, wherein the aggregate façade view comprises visual pixels from at least one identified camera;
photo-texturing one of the respective façade slice or geometric model with the aggregate façade view to generate a visual 3-D representation of the building; and
generating a training dataset by capturing images of the visual 3-D representation from a plurality of perturbed positions.

### Claim 15 (depends on 14)

The method of claim 14, wherein generating the respective façade slice comprises isolating a respective façade as a two-dimensional representation in a three-dimensional coordinate system of the geometric model.

### Claim 16 (depends on 15)

The method of claim 15, wherein isolating the respective façade further comprises applying a bounding box to the respective façade of the geometric model.

### Claim 17 (depends on 16)

The method of claim 16, wherein identifying the respective camera that observes a respective façade slice further comprises:
reprojecting the bounding box into the plurality of images; and
recording each camera that observes the bounding box in its associated image.

### Claim 18 (depends on 14)

The method of claim 14, wherein the visibility mask is a classification indicating one or more pixels related to a façade slice are observed from the identified camera.

### Claim 19 (depends on 14)

The method of claim 14, wherein generating the aggregate façade view further comprises generating one or more visual data templates for the at least one identified camera, wherein each visual data template is a modified image comprising visual pixel data of a façade slice according to visibility mask of the at least one identified camera.

### Claim 20 (depends on 19)

The method of claim 19, wherein each visual data template is transformed to a common perspective.

### Claim 21 (depends on 20)

The method of claim 20, wherein the common perspective is an orientation with a plane of the modified image orthogonal to an optical axis of a virtual camera displaying the modified image.

### Claim 22 (depends on 20)

The method of claim 20, wherein generating the respective façade slice comprises isolating a respective façade as a two-dimensional representation in a three-dimensional coordinate system of the geometric model, wherein isolating the respective façade further comprises applying a bounding box to the respective façade of the geometric model, and wherein the common perspective aligns associated boundaries for the bounding box.

### Claim 23 (depends on 14)

The method of claim 14, wherein generating the aggregate façade view further comprises selecting, for each façade slice, a base visual data template, wherein the base visual data template has a largest observed pixel count among each visual data template for the respective façade slice.

### Claim 24 (depends on 23)

The method of claim 23, further comprising cumulating the visual pixels of the base visual data template with visual pixels from additional visual data templates for the respective façade slice.

### Claim 25 (depends on 14)

The method of claim 14, wherein each perturbed position of the visual 3-D representation is relative to one or more cameras of the camera solution.

### Claim 26 (depends on 14)

The method of claim 14, wherein each perturbed position of the visual 3-D representation is relative to a corresponding perspective of a virtual camera.

### Claim 27 (depends on 14)

The method of claim 14, wherein a perturbed position of the visual 3-D representation includes one or more of translating, rotating, zooming, or combinations thereof.

### Claim 28 (depends on 14)

The method of claim 14, further comprising identifying feature matches across captured images.

### Claim 29

A computer system for generating training data for a building structure, comprising:
one or more processors, including a general purpose processor and a graphics processing unit (GPU);
a display; and
memory;
wherein the memory stores one or more programs configured for execution by the one or more processors, and the one or more programs comprising instructions for:
obtaining a geometric model of a building that includes a camera solution and a plurality of images used to generate the geometric model;
for a plurality of facades of the geometric model:
applying a minimum bounding box to a respective façade to obtain a respective façade slice that is a 2-D plane represented in a 3-D coordinate system of the geometric model; and
projecting visual data of at least one camera in the camera solution that viewed the respective façade onto a visibility mask associated with the respective facade slice;
photo-texturing the projected visual data to one of the respective facade slice or the geometric model to generate a visual 3-D representation of the building; and
generating a training dataset by perturbing the visual 3-D representation.

### Claim 30

A non-transitory computer readable storage medium storing one or more programs configured for execution by a computer system having a display, one or more processors including a general purpose processor and a graphical processing unit (GPU), the one or more programs comprising instructions for:
obtaining a geometric model of a building that includes a camera solution and a plurality of images used to generate the geometric model;
for a plurality of facades of the geometric model:
applying a minimum bounding box to a respective façade to obtain a respective façade slice that is a 2-D plane represented in a 3-D coordinate system of the geometric model; and
projecting visual data of at least one camera in the camera solution that viewed the respective façade onto a visibility mask associated with the respective facade slice;
photo-texturing the projected visual data to one of the respective facade slice or the geometric model to generate a visual 3-D representation of the building; and
generating a training dataset by perturbing the visual 3-D representation.
