# US 2019/0318547 A1 · System and method for dense, large scale scene reconstruction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318547 A1
- Document ID: US20190318547A1
- Status: Published application
- Kind: A1
- Application number: 16/455,923
- Filed: 2019-06-28
- Published: 2019-10-17
- Earliest claimed: 2016-09-19
- Assignees: Occipital, Inc.
- Applicants: Occipital, Inc.
- Inventors: Jeffrey Powers; Nicolas Burrus; Yuping Lin; Paul Schroeder
- Web page: https://patentgrep.com/patent/US20190318547A1

## CPC

- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 7/536: from perspective effects, e.g. by using vanishing points
- G06T 7/85: {Stereo camera calibration}
- G06T 15/205: {Image-based rendering}
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 2215/16: Using real world measurements to influence rendering
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system configured to improve the operations associated with generating virtual representations on limited resources of a mobile device. In some cases, the system may utilize viewpoint bundles that include collection of image data with an associated pose in relative physical proximity to each other to render a virtual scene. In other cases, the system may utilize 2.5D manifolds including 2D image data and a weighted depth value to render the 3D environment.

## Claims

### Claim 1

A method comprising:
receiving a first voxel block of a first size and a second voxel block of a second size, the second size coarser than the first size;
generating a super voxel block including at least the first voxel block and the second voxel block;
integrating adjacent voxel blocks of the super voxel block having the second size using one marching pyramid into a mesh;
integrating up to four adjacent voxel blocks of the first size with a share-plane of the super voxel block using four marching pyramids into the mesh;
integrating voxel blocks of the first size and voxel blocks of the second size sharing three edges within the super voxel block using two marching pyramids and one marching tetrahedral into the mesh;
integrating voxel blocks of the first size and voxel blocks of the second size sharing two edges within the super voxel block using three marching pyramids, two marching pyramids, and two marching tetrahedras into the mesh; and
integrating voxel blocks of the first size and voxel blocks of the second size sharing one edges within the super voxel block using four marching pyramids into the mesh.

### Claim 2 (depends on 1)

The method as recited in claim 1, wherein the voxel blocks of the second size corresponding to a surface in the physical environment at a distance greater than a threshold from a first pose and the voxel blocks of the first size corresponding to at least a second surface in the physical environment at a distance less than the threshold from the first pose.

### Claim 3 (depends on 1)

The method as recited in claim 1, further comprising rendering a virtual environment based at least in part on the TSDF.

### Claim 4 (depends on 1)

The method as recited in claim 1, wherein the mesh is a global triangle mesh representing a physical environment.

### Claim 5 (depends on 1)

The method as recited in claim 1, further comprising:
storing a first vertex of a surface associated with a first voxel block within a first local vertex array corresponding to the first voxel block; and
computing an index mapping the first vertex to a first entry in an edge array and the first entry in the edge array to a local vertex array.

### Claim 6 (depends on 5)

The method as recited in claim 5, further comprising:
identifying a second vertex associated with a second voxel block, the second voxel block adjacent to the first voxel block;
mapping the second vertex to a second entry in the edge array; and
determining that an edge corresponding to the entry in the edge array belongs to the first voxel block. to the particular voxel block.

### Claim 7

A system comprising:
a display for presenting a virtual environment to a user;
one or more image components for capturing image data associated with a physical environment surrounding the user;
one or more processors;
non-transitory computer-readable media storing computer-executable instructions, which when executed by the one or more processors cause the one or more processors to perform operations including:
receiving a first voxel block of a first size and a second voxel block of a second size;
generating a super voxel block including at least the first voxel block and the second voxel block;
integrating adjacent voxel blocks of the super voxel block having the second size using a first set of functions into a mesh;
integrating up to four adjacent voxel blocks of the first size with a share-plane of the super voxel block using a second set of functions into the mesh;
integrating voxel blocks of the first size and voxel blocks of the second size sharing three edges within the super voxel block using a third set of functions into the mesh;
integrating voxel blocks of the first size and voxel blocks of the second size sharing two edges within the super voxel block using a fourth set of functions into the mesh; and
integrating voxel blocks of the first size and voxel blocks of the second size sharing one edges within the super voxel block using a fifth set of functions into the mesh.

### Claim 8 (depends on 7)

The system as recited in claim 7, wherein the voxel blocks of the second size corresponding to a surface in the physical environment at a distance greater than a threshold from a first pose and the voxel blocks of the first size corresponding to at least a second surface in the physical environment at a distance less than the threshold from the first pose.

### Claim 9 (depends on 7)

The system as recited in claim 7, wherein the non-transitory computer-readable media storing computer-executable instructions, which when executed by the one or more processors cause the one or more processors to perform additional operations including comprising rendering a virtual environment based at least in part on the TSDF.

### Claim 10 (depends on 7)

The system as recited in claim 7, wherein the non-transitory computer-readable media storing computer-executable instructions, which when executed by the one or more processors cause the one or more processors to perform additional operations including:
storing a first vertex of a surface associated with a first voxel block within a first local vertex array corresponding to the first voxel block; and
computing an index mapping the first vertex to a first entry in an edge array and the first entry in the edge array to a local vertex array.

### Claim 11 (depends on 10)

The system as recited in claim 10, wherein the non-transitory computer-readable media storing computer-executable instructions, which when executed by the one or more processors cause the one or more processors to perform additional operations including:
identifying a second vertex associated with a second voxel block, the second voxel block adjacent to the first voxel block;
mapping the second vertex to a second entry in the edge array; and
determining that an edge corresponding to the entry in the edge array belongs to the first voxel block. to the particular voxel block.

### Claim 12 (depends on 11)

The system as recited in claim 11, wherein the second size coarser than the first size.

### Claim 13 (depends on 7)

The system as recited in claim 7, wherein using the first set of functions includes using one marching pyramid.

### Claim 14 (depends on 7)

The system as recited in claim 7, wherein using the second set of functions includes using four marching pyramids.

### Claim 15 (depends on 7)

The system as recited in claim 7, wherein using the third set of functions includes using two marching pyramids and one marching tetrahedral.

### Claim 16 (depends on 7)

The system as recited in claim 7, wherein using the fourth set of functions includes using three marching pyramids, two marching pyramids, and two marching tetrahedras.

### Claim 17 (depends on 7)

The system as recited in claim 7, wherein using the fifth set of functions includes using four marching pyramids.

### Claim 18

A non-transitory computer-readable media storing computer-executable instructions, which when executed by the one or more processors cause the one or more processors to perform operations including:
generating a super voxel block including at least the first voxel block and the second voxel block;
integrating voxel blocks of the first size and voxel blocks of the second size sharing three edges within the super voxel block using two marching pyramids and one marching tetrahedral into the mesh;
integrating voxel blocks of the first size and voxel blocks of the second size sharing two edges within the super voxel block using three marching pyramids, two marching pyramids, and two marching tetrahedras into the mesh; and
integrating voxel blocks of the first size and voxel blocks of the second size sharing one edges within the super voxel block using four marching pyramids into the mesh.

### Claim 19 (depends on 18)

The non-transitory computer-readable media storing as recited in claim 18, storing additional computer-executable instructions, which when executed by the one or more processors cause the one or more processors to perform additional operations including integrating adjacent voxel blocks of the super voxel block having the second size using one marching pyramid into a mesh.

### Claim 20 (depends on 18)

The non-transitory computer-readable media storing as recited in claim 18, storing additional computer-executable instructions, which when executed by the one or more processors cause the one or more processors to perform additional operations including integrating adjacent voxel blocks of the super voxel block having the second size using one marching pyramid into a mesh. integrating up to four adjacent voxel blocks of the first size with a share-plane of the super voxel block using four marching pyramids into the mesh.
