# US 2019/0244414 A1 · Efficient Volumetric Reconstruction with Depth Sensors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244414 A1
- Document ID: US20190244414A1
- Status: Published application
- Kind: A1
- Application number: 16/316,482
- Filed: 2017-07-13
- Published: 2019-08-08
- Earliest claimed: 2016-07-13
- Assignees: Naked Labs Austria GMBH
- Applicants: Naked Labs Austria GMBH
- Inventors: William Nguyen
- Web page: https://patentgrep.com/patent/US20190244414A1

## CPC

- G06T 15/08: Volume rendering
- G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration
- G06T 7/50: Depth or shape recovery
- G06T 7/55: from multiple images
- G06T 2200/04: involving 3D image data
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method for real-time volumetric 3-D reconstruction of an object uses at least one depth sensor camera. In a preparation step, voxels are collected in a reconstructed scene depending on a new depth map frame, and the collected voxels are cached in order to perform an update of the reconstructed scene. In an integration step, the collected and cached voxels of the preparation step are updated with a newly captured depth map frame. However, the preparation step, and the integration step are separated from each other so that both steps can be carried out in parallel, and at the same time.

## Claims

### Claim 1

A method for real-time volumetric 3D reconstruction of an object using at least one depth sensor camera, the method comprising:
performing a preparation step of a new depth map frame, in which voxels are collected in a reconstructed scene depending on the new depth map frame and in which the collected voxels are cached in order to perform an update of the reconstructed scene; and
performing an integration step, in which the collected and cached voxels of the preparation step are updated with a newly captured depth map frame;
wherein the preparation step and the integration step are separated from each other so that both the preparation and integration steps can be carried out in parallel at the same time.

### Claim 2 (depends on 1)

Method according to claim 1, wherein in the preparation step a first cast ray is generated, which first cast ray extends from a camera center through an associated 3D position of a pixel on a camera plane, wherein in the preparation step a first depth interval is determined on the first cast ray, wherein the first depth interval is determined on the first cast ray depending on the new depth map frame and depending on a depth value of the associated pixel.

### Claim 3 (depends on 1)

Method according to claim 1, wherein the preparation step includes a search step in which are searched an existing voxel and a non-existing voxel that is adjacent the existing voxel, which are hit by the first cast ray in the first depth interval.

### Claim 4 (depends on 1)

Method according to claim 1, wherein in the preparation step the existing voxel and the non-existing voxel are searched on each pixel by checking the respective first depth interval along the first cast ray.

### Claim 5 (depends on 1)

Method according to claim 1, wherein in the preparation step multiple search steps are parallelized such that each thread does searching on one pixel.

### Claim 6 (depends on 1)

Method according to claim 1, wherein as a result of the preparation step a list of existing voxels and non-existing voxels, which are close to the observed depth map, is prepared.

### Claim 7 (depends on 1)

Method according to claim 1, wherein in the integration step the update starts by re-projecting a voxel origin position of a reconstructed voxel of the reconstructed scene to the camera plane.

### Claim 8 (depends on 1)

Method according to claim 1, wherein a second cast ray is generated for re-projecting the voxel origin position of the reconstructed voxel to the camera plane, wherein the second cast ray extends from the camera center through the associated voxel origin position of the reconstructed voxel.

### Claim 9 (depends on 1)

Method according to claim 1, further comprising the steps of: determining a re-projected location of the voxel origin on the camera plane, then determining an associated depth value of an associated pixel of the depth map frame, and using the determined depth value used to update a Signed Distance Function of the corresponding voxel.

### Claim 10 (depends on 1)

Method according to claim 1, wherein the integration step is parallelized such that each thread updates separate reconstructed voxels of the reconstructed scene.

### Claim 11 (depends on 1)

Method according to claim 1, wherein an indexing mechanism is used in which a mapping array is predefined, wherein the mapping array maps the 3D voxel origin position of at least one reconstructed voxel of the scene to a pre-allocated array of voxel blocks and
wherein on the basis of experience values that have been experimentally derived, a number of reconstructed surface voxels is predicted and the array of reconstructed voxel blocks is pre-allocated with this number.

### Claim 12 (depends on 1)

Method according to claim 1, wherein for parallelizing the preparation step and the integration step, the non-existing voxels are successively updated to the array of the reconstructed voxel blocks.

### Claim 13 (depends on 1)

Method according to claim 1, wherein the integration step concurrently processes the list of existing and non-existing voxels.

### Claim 14 (depends on 1)

Method according to claim 1, further comprising: a ray-casting step, in which the depth map input is exploited, wherein a second depth interval is determined on a third cast ray by using an initial observed depth map frame where the depth value of the associated pixel is known, wherein voxels are searched along the second depth interval, which are hit by the third cast ray, and wherein a surface point along the third cast ray is interpolated using a Signed Distance Function values of the founded and neighboring voxels.

### Claim 15 (depends on 1)

3D body scanner for real-time volumetric 3D reconstruction of an object, the 3D body scanner comprising:
at least one depth sensor; and
a processor, which is designed to operate with a method as set forth in claim 1 and is connected to the at least one depth sensor.
