# US 2021/0065429 A1 · Rendering operations using sparse volumetric data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0065429 A1
- Document ID: US20210065429A1
- Status: Published application
- Kind: A1
- Application number: 16/994,511
- Filed: 2020-08-14
- Published: 2021-03-04
- Earliest claimed: 2016-08-19
- Assignees: Movidius Limited
- Applicants: Movidius Ltd.
- Inventors: Sam Caulfield; David Macdara Moloney; Gary Garfield Barrington Baugh
- Web page: https://patentgrep.com/patent/US20210065429A1

## CPC

- G01C 21/20: Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence)
- G01C 21/30: Map- or contour-matching
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G05D 1/246: using environment maps, e.g. simultaneous localisation and mapping [SLAM]
- G05D 1/617: Safety or protection, e.g. defining protection zones around obstacles or avoiding hazards (arrangements for controlling the position or course of two or more vehicles for avoiding collisions therebetween G05D1/693; arrangements for reacting to or preventing system or operator failure G05D1/80)
- G06F 9/30029: {Logical and Boolean instructions, e.g. XOR, NOT}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/09: Supervised learning
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 7/579: from motion
- G06T 7/593: from stereo images
- G06T 7/75: {involving models}
- G06T 15/06: Ray-tracing
- G06T 15/08: Volume rendering
- G06T 17/005: {Tree description, e.g. octree, quadtree}
- G06T 17/05: Geographic models
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 20/13: Satellite images
- G06V 20/17: taken from planes or by drones
- G06V 20/64: Three-dimensional [3D] objects
- G06N 3/048: Activation functions
- G06T 2200/04: involving 3D image data
- G06T 2200/28: involving image processing hardware
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30244: Camera pose
- G06T 2207/30261: Obstacle
- G06T 2210/08: Bandwidth reduction
- G06T 2210/36: Level of detail
- G06T 2219/004: Annotating, labelling
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A ray is cast into a volume described by a volumetric data structure, which describes the volume at a plurality of levels of detail. A first entry in the volumetric data structure includes a first set of bits representing voxels at a lowest one of the plurality of levels of detail, and values of the first set of bits indicate whether a corresponding one of the voxels is at least partially occupied by respective geometry. A set of second entries in the volumetric data structure describe voxels at a second level of detail, which represent subvolumes of the voxels at the first lowest level of detail. The ray is determined to pass through a particular subset of the voxels at the first level of detail and at least a particular one of the particular subset of voxels is determined to be occupied by geometry.

## Claims

### Claim at position 1

-65. (canceled)

### Claim 66

An apparatus comprising:
a processor;
a first memory to store a volumetric data structure corresponding to the particular volume, wherein the volumetric data structure describes the particular volume at a plurality of levels of detail, a first entry in the volumetric data structure comprises a first set of bits representing voxels at a first one of the plurality of levels of detail, values of the first set of bits indicate whether a corresponding one of the voxels is at least partially occupied by respective geometry, wherein the volumetric data structure further comprises a number of second entries representing voxels at a second level of detail higher than the first level of detail, and the voxels at the second level of detail represent subvolumes of the voxels at the first level of detail;
a second memory; and
a memory manager to:
load a portion of the volumetric data structure into the second memory based on the portion of the volumetric data structure representing a particular subset of the plurality of levels of detail, wherein the portion of the volumetric data structure is for use by the processor to perform an operation.

### Claim 67 (depends on 66)

The apparatus of claim 66, wherein the processor comprises a plurality of cores.

### Claim 68 (depends on 67)

The apparatus of claim 67, wherein the operation comprises a plurality of jobs and the plurality of cores are to each perform a respective on of the plurality of jobs.

### Claim 69 (depends on 68)

The apparatus of claim 68, wherein the plurality of jobs are performed at least partially in parallel by the plurality of cores.

### Claim 70 (depends on 66)

The apparatus of claim 66, wherein the operation comprises raycasting.

### Claim 71 (depends on 70)

The apparatus of claim 70, wherein the raycasting is to be performed in association with rendering of an image based on the volumetric data structure.

### Claim 72 (depends on 70)

The apparatus of claim 70, wherein raycasting comprises determining whether a ray intersects with particular geometry in the particular volume.

### Claim 73 (depends on 72)

The apparatus of claim 72, further comprising rendering logic to determine a reflection of the ray off the particular geometry.

### Claim 74 (depends on 73)

The apparatus of claim 73, wherein the ray models an audio ray, and the reflection of the ray is determined based on a reflection coefficient associated with the particular geometry.

### Claim 75 (depends on 74)

The apparatus of claim 74, wherein the reflection coefficient is identified in the volumetric data structure.

### Claim 76 (depends on 74)

The apparatus of claim 74, wherein the rendering logic is to generate a reverberation model corresponding to the geometry within the particular volume based on the determined reflections of a plurality of rays cast into the particular volume, and the plurality of rays comprise the ray.

### Claim 77 (depends on 74)

The apparatus of claim 74, further comprising a filter generator to generate a finite impulse response filter corresponding to the reverberation model.

### Claim 78 (depends on 66)

The apparatus of claim 66, where the number of second entries corresponds to a number of bits in the first set of bits with values indicating that a corresponding voxel at the first level of detail is at least partially occupied with geometry.

### Claim 79 (depends on 66)

The apparatus of claim 66, wherein the second memory comprises on-board memory of the processor.

### Claim 80

A method comprising:
determining an operation to perform using a volumetric data structure corresponding to the particular volume, wherein the volumetric data structure describes the particular volume at a plurality of levels of detail, a first entry in the volumetric data structure comprises a first set of bits representing voxels at a first one of the plurality of levels of detail, values of the first set of bits indicate whether a corresponding one of the voxels is at least partially occupied by respective geometry, wherein the volumetric data structure further comprises a number of second entries representing voxels at a second level of detail higher than the first level of detail, and the voxels at the second level of detail represent subvolumes of the voxels at the first level of detail;
loading a portion of the volumetric data structure into on-board memory of a processor device based on the portion of the volumetric data structure representing a particular subset of the plurality of levels of detail; and
using the portion of the volumetric data structure in the on-board memory to perform the operation at the processor.

### Claim 81 (depends on 80)

The method of claim 80, wherein the operation comprises raycasting.

### Claim 82

At least one machine accessible storage medium having instructions stored thereon, the instructions when executed on a machine, cause the machine to:
access a volumetric data structure from memory corresponding to the particular volume, wherein the volumetric data structure describes the particular volume at a plurality of levels of detail, a first entry in the volumetric data structure comprises a first set of bits representing voxels at a first one of the plurality of levels of detail, the first level of detail comprises the lowest level of detail in the plurality of levels of detail, values of the first set of bits indicate whether a corresponding one of the voxels is at least partially occupied by respective geometry, wherein the volumetric data structure further comprises a number of second entries representing voxels at a second level of detail higher than the first level of detail, and the voxels at the second level of detail represent subvolumes of the voxels at the first level of detail; and
loading a portion of the volumetric data structure into on-board memory of a processor device based on the portion of the volumetric data structure representing a particular subset of the plurality of levels of detail; and
causing the processor device to perform the operation using the portion of the volumetric data structure.

### Claim 83 (depends on 82)

The storage medium of claim 82, wherein the instructions, when executed further cause the machine to:
identify that the operation is to be performed at a higher level of detail than the particular subset of the plurality of levels of detail; and
loading another portion of the volumetric data structure into the on-board memory, wherein the other portion of the volumetric data structure represents the higher level of detail.

### Claim 84 (depends on 83)

The storage medium of claim 83, wherein the operation comprises raycasting.

### Claim 85 (depends on 84)

The storage medium of claim 84, wherein the raycasting comprises casting a plurality of rays through the particular volume to determine whether subvoxels of the particular volume are occupied, and the processor device comprises a plurality of cores,
wherein the instructions, when executed, further cause the machine to assign raycasting of a first subset of the plurality of rays to be performed by a first one of the plurality of cores, and assign raycasting of a second subset of the plurality of rays to be performed by a second one of the plurality of cores.
