# US 2026/0073622 A1 · World tracked planes from volumetric geometry

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073622 A1
- Document ID: US20260073622A1
- Status: Published application
- Kind: A1
- Application number: 18/883,305
- Filed: 2024-09-12
- Published: 2026-03-12
- Earliest claimed: 2024-09-12
- Assignees: Snap Inc.
- Applicants: Snap Inc.
- Inventors: Tobias Maier; Jakob Zillner; Olha Borys
- Web page: https://patentgrep.com/patent/US20260073622A1

## CPC

- G06T 7/579: from motion
- G06T 15/10: Geometric effects
- G06T 7/50: Depth or shape recovery
- G06T 15/08: Volume rendering
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system is disclosed, including a processor and a memory. The memory stores instructions that, when executed by the processor, configure the system to perform operations. Depth estimates are used to generate distance values by applying a signed distance function to the depth estimates. A 3D representation of the environment is generated using the distance estimates. Local planes are fit to the 3D representation, and larger planes are generated by merging local planes using predefined criteria such as surface normal agreement. Larger planes are dynamically updated or removed in response to updated depth estimates.

## Claims

### Claim 1

A method for detecting planes in an augmented reality environment, the method comprising:
generating a plurality of depth estimates from one or more of: a series of posed camera images and a dataset of visual inertial odometry (VIO) points;
determining a plurality of distance values by applying a signed distance function to the plurality of depth estimates;
configuring a voxel representation of the plurality of distance values;
fitting a plurality of local planes to blocks of voxels in the voxel representation, wherein each block comprises multiple voxels;
generating a first larger plane from merging a first subset of the plurality of local planes based on predefined criteria; and
generating a second larger plane from merging a second subset of the plurality of local planes based on the predefined criteria, wherein the first subset is distinct from the second subset, and wherein a first normal vector of the first larger plane is different from a second normal vector of the second larger plane.

### Claim 2 (depends on 1)

The method of claim 1, wherein the method further comprises:
generating a plurality of updated depth estimates based on an update to one or more of:
the series of posed camera images and the dataset of VIO points;
determining a plurality of updated distance values by applying the signed distance function to the plurality of updated depth estimates;
updating the voxel representation with the plurality of updated distance values; and
removing at least one of the first larger plane and the second larger plane based at least on updates to the voxel representation.

### Claim 3 (depends on 2)

The method of claim 2, wherein the method further comprises:
fitting a plurality of updated local planes to blocks of voxels in the updated voxel representation; and
generating a third larger plane from merging a first subset of the plurality of updated local planes based on the predefined criteria.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining the plurality of distance values by applying the signed distance function to the plurality of depth estimates comprises:
continuously generating additional plurality of depth estimates based on continuous updates to one or more of: the series of posed camera images and the dataset of VIO points; and
averaging the additional plurality of depth estimates into the voxel representation.

### Claim 5 (depends on 1)

The method of claim 1, wherein the signed distance function is a truncated signed distance function.

### Claim 6 (depends on 1)

The method of claim 1, wherein generating the first larger plane from merging the first subset of the plurality of local planes comprises a comparison of a surface normal vector of each local plane with a neighboring local plane.

### Claim 7 (depends on 1)

The method of claim 1, wherein the predefined criteria comprise at least one of a similarity threshold for surface normal vectors of the plurality of local planes and a root mean square error below a predetermined threshold.

### Claim 8 (depends on 1)

The method of claim 1, wherein a block of voxels used in fitting the plurality local planes to blocks of voxels in the voxel representation comprises 8×8×8 voxels.

### Claim 9 (depends on 1)

The method of claim 1, wherein a size of a block of voxels used in fitting the plurality of local planes to blocks of voxels is based on a distance in the voxel representation.

### Claim 10 (depends on 9)

The method of claim 9, wherein the size of the block of voxels is larger than 8×8×8 voxels when the distance in the voxel representation is above a first threshold.

### Claim 11 (depends on 9)

The method of claim 9, wherein the size of the block of voxels is smaller than 8×8×8 voxels when the distance in the voxel representation is below a second threshold.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
extending the first larger plane to neighboring blocks of voxels when the neighboring blocks of voxels meet the predefined criteria.

### Claim 13 (depends on 1)

The method of claim 1, further comprising sampling points from the first larger plane.

### Claim 14 (depends on 13)

The method of claim 13, further comprising using the sampled points to improve accuracy of the dataset of VIO points.

### Claim 15 (depends on 13)

The method of claim 13, further comprising using the sampled points to improve accuracy of the plurality of depth estimates.

### Claim 16

A system for detecting planes in an augmented reality environment, the system comprising:
one or more processors;
a memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
generating depth estimates from one or more of: a series of posed camera images and a dataset of visual inertial odometry (VIO) points;
generating distance values by applying a signed distance function to the depth estimates;
configuring a voxel representation of the distance values;
fitting local planes to blocks of voxels in the voxel representation, wherein each block comprises multiple voxels;
generating larger planes by merging one or more of the local planes based on predefined criteria; and
at least one of dynamically update or dynamically remove one or more of the larger planes in response to generating one or more updated depth estimates.

### Claim 17 (depends on 16)

The system of claim 16, wherein the operations further comprise:
continuously generate additional depth estimates based on continuous updates to one or more of: the series of posed camera images and the dataset of VIO points; and
averaging the additional depth estimates into the voxel representation.

### Claim 18 (depends on 16)

The system of claim 16, wherein the predefined criteria comprise at least one of a similarity threshold for surface normal vectors of the local planes and a root mean square error below a predetermined threshold.

### Claim 19 (depends on 16)

The system of claim 16, wherein the operations further comprise:
sampling points from at least one of the larger planes; and
using the sampled points to improve at least one of: accuracy of the dataset of VIO points and accuracy of the depth estimates.

### Claim 20

A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a processor of a system, cause the system to perform operations comprising:
generating a plurality of depth estimates from one or more of: a series of posed camera images and a dataset of visual inertial odometry (VIO) points;
determining a plurality of distance values by applying a signed distance function to the plurality of depth estimates;
configuring a voxel representation with the plurality of distance values;
fitting a plurality of local planes to blocks of voxels in the voxel representation, wherein each block comprises multiple voxels;
generating a first larger plane from merging a first subset of the plurality of local planes based on predefined criteria;
generating a second larger plane from merging a second subset of the plurality of local planes based on the predefined criteria, wherein the first subset is distinct from the second subset, and wherein a first normal vector of the first larger plane is different from a second normal vector of the second larger plane; and
at least one of dynamically updating or dynamically removing one or more of the first larger plane and the second larger plane in response to generating one or more updated depth estimates.
