# US 2019/0378332 A1 · Systems and methods for filling holes in a virtual reality model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378332 A1
- Document ID: US20190378332A1
- Status: Published application
- Kind: A1
- Application number: 16/426,035
- Filed: 2019-05-30
- Published: 2019-12-12
- Earliest claimed: 2018-06-06
- Assignees: Ke.com (Beijing) Technology Co., Ltd.
- Applicants: Ke.com (Beijing) Technology Co., Ltd
- Inventors: Xin Sun; Zhe Xie
- Web page: https://patentgrep.com/patent/US20190378332A1

## CPC

- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 15/04: Texture mapping
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes
- G06T 2219/2021: Shape modification
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems and methods for constructing a three-dimensional virtual reality model of a property are disclosed. An exemplary system includes a storage device configured to store point cloud data and texture information captured of the property and at least one processor. The processor constructs a structure polygon mesh based on the point cloud data and constructs a textured polygon mesh by mapping the texture information to the structure polygon mesh. The processor further identifies a hole in the textured polygon mesh based on the mapping and determine an area of the hole is below a predetermined threshold. The processor then identifies boundary polygons in the textured polygon mesh that are contiguous to the hole and estimates texture information of the hole, and repair the textured polygon mesh using the estimated texture information of the hole polygons.

## Claims

### Claim 1

A system for constructing a three-dimensional virtual reality model of a property, comprising:
a storage device configured to store point cloud data and texture information captured of the property; and
at least one processor configured to:
construct a structure polygon mesh of the three-dimensional virtual reality model based on the point cloud data, the structure polygon mesh including a first set of polygons;
construct a textured polygon mesh of the three-dimensional virtual reality model by mapping the texture information to the structure polygon mesh, the textured polygon mesh including a second set of polygons;
identify a hole in the textured polygon mesh based on the mapping, the hole including a plurality of hole polygons;
determine an area of the hole is below a predetermined threshold;
identify boundary polygons in the second set of polygons that are contiguous to the hole;
estimate texture information of the hole polygons based on texture information of the boundary polygons and respective distances between the hole polygons and the boundary polygons; and
repair the textured polygon mesh using the estimated texture information of the hole polygons.

### Claim 2 (depends on 1)

The system of claim 1, wherein the first set of polygons, the second set of polygons, and the hole polygons are triangles.

### Claim 3 (depends on 1)

The system of claim 1, wherein to identify the hole in the textured polygon mesh, the at least one processor is configured to identify a third set of polygons from the first set of polygons that do not have corresponding textured polygons in the second set of polygons.

### Claim 4 (depends on 3)

The system of claim 3, the hole polygons in the hole correspond to the third set of polygons.

### Claim 5 (depends on 1)

The system of claim 1, wherein to estimate texture information for the hole polygons, the at least one processor is further configured to:
determine pixel values for vertices of each hole polygon; and
determine pixel values inside each hole polygon based on the pixel values of its vertices.

### Claim 6 (depends on 5)

The system of claim 5, wherein to determine the pixel value for a vertex of a hole polygon, the at least one processor is further configured to:
determine distances between the vertex and vertices of boundary polygons; and
calculate a weighted sum of pixel values of the vertices of boundary polygons, wherein the pixel values of the boundary vertices are inversely weighted by the distances between the vertex of the hole polygon and vertices of the respective boundary vertices.

### Claim 7 (depends on 5)

The system of claim 5, wherein to determine pixel values inside each hole polygon, the at least one processor is further configured to perform data fusion on the pixel values of the vertices of the hole polygon based on a Gaussian process.

### Claim 8 (depends on 1)

The system of claim 1, wherein to determine the area of the hole, the at least one processor is further configured to:
determine areas of the hole polygons in the hole; and
determine the area of the hole as a sum of the areas of the hole polygons.

### Claim 9 (depends on 1)

The system of claim 1, wherein the structure polygon mesh is constructed from the point cloud data using Poisson surface reconstruction.

### Claim 10

A computer-implemented method for constructing a three-dimensional virtual reality model of a property, comprising:
constructing, by at least one processor, a structure polygon mesh of the three-dimensional virtual reality model based on point cloud data captured of the property, the structure polygon mesh including a first set of polygons;
constructing, by the at least one processor, a textured polygon mesh of the three-dimensional virtual reality model by mapping texture information to the structure polygon mesh, the textured polygon mesh including a second set of polygons;
identifying, by the at least one processor, a hole in the textured polygon mesh based on the mapping, the hole including a plurality of hole polygons;
determining, by the at least one processor, an area of the hole is below a predetermined threshold;
identifying, by the at least one processor, boundary polygons in the second set of polygons that are contiguous to the hole;
estimating, by the at least one processor, texture information of the hole polygons based on the texture information of the boundary polygons and respective distances between the hole polygons and the boundary polygons; and
repairing, by the at least one processor, the textured polygon mesh using the estimated texture information of the hole polygons.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein the first set of polygons, the second set of polygons, and the hole polygons are triangles.

### Claim 12 (depends on 10)

The computer-implemented method of claim 10, wherein identifying the hole in the textured polygon mesh further includes identifying a third set of polygons from the first set of polygons that do not have corresponding textured polygons in the second set of polygons.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, the hole polygons in the hole correspond to the third set of polygons.

### Claim 14 (depends on 10)

The computer-implemented method of claim 10, wherein estimating texture information for the hole polygons further include:
determining pixel values for vertices of each hole polygon; and
determining pixel values inside each hole polygon based on the pixel values of its vertices.

### Claim 15 (depends on 14)

The computer-implemented method of claim 14, wherein determining the pixel value for a vertex of a hole polygon further includes:
determining distances between the vertex and vertices of boundary polygons; and
calculating a weighted sum of pixel values of the vertices of boundary polygons, wherein the pixel values of the boundary vertices are inversely weighted by the distances between the vertex of the hole polygon and vertices of the respective boundary vertices.

### Claim 16 (depends on 14)

The computer-implemented method of claim 14, wherein determining pixel values inside each hole polygon further includes performing data fusion on the pixel values of the vertices of the hole polygon based on a Gaussian process.

### Claim 17 (depends on 10)

The computer-implemented method of claim 10, wherein determining the area of the hole further includes:
determining areas of the hole polygons in the hole; and
determine the area of the hole as a sum of the areas of the hole polygons.

### Claim 18 (depends on 10)

The computer-implemented method of claim 10, wherein the structure polygon mesh is constructed from the point cloud data using Poisson surface reconstruction.

### Claim 19

A non-transitory computer-readable medium having stored thereon computer instructions, when executed by at least one processor, perform a method for constructing a three-dimensional virtual reality model of a property, comprising:
constructing a structure polygon mesh of the three-dimensional virtual reality model based on point cloud data captured of the property, the structure polygon mesh including a first set of polygons;
constructing a textured polygon mesh of the three-dimensional virtual reality model by mapping the texture information to the structure polygon mesh, the textured polygon mesh including a second set of polygons;
identifying a hole in the textured polygon mesh based on the mapping, the hole including a plurality of hole polygons;
determining an area of the hole is below a predetermined threshold;
identifying boundary polygons in the second set of polygons that are contiguous to the hole;
estimating texture information of the hole polygons based on texture information of the boundary polygons and respective distances between the hole polygons and the boundary polygons; and
repairing the textured polygon mesh using the estimated texture information of the hole polygons.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, wherein estimating texture information of the hole polygons further includes:
determining pixel values for vertices of each hole polygon as a weighted sum of pixel values of the vertices of boundary polygons, wherein the pixel values of the boundary vertices are inversely weighted by distances between the vertices of the hole polygon and vertices of the respective boundary vertices; and
determining pixel values inside each hole polygon based on the pixel values of its vertices based on a Gaussian process.