# US 12,762,850 B2 · Automated object protrusion loss adjustment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,762,850 B2
- Document ID: US12762850B2
- Status: Granted
- Kind: B2
- Application number: 18/956,093
- Filed: 2024-11-22
- Published: 2026-05-28
- Granted: 2026-10-13
- Earliest claimed: 2024-11-22
- Assignees: Zoox, Inc.
- Applicants: Zoox, Inc.
- Inventors: Arthur Daniel Costea; Rajendramayavan Rajendran Sathyam
- Web page: https://patentgrep.com/patent/US12762850B2

## CPC

- G06V 10/776: Validation; Performance evaluation
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/64: Three-dimensional [3D] objects
- G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Techniques for determining labels and occupancy data for voxels and pixels representing object protrusions are disclosed. The occupancy status of voxels surrounding an occupied voxel is determined and used to determine the occupancy density of the occupied voxel. The loss for the occupied voxel is adjusted inversely proportionately to the occupancy density. The adjusted-loss voxel is used to train a machine-learned model to detect objects in an environment and, specifically, to more accurately detect objects having protrusions that may otherwise not be associated with the object. This model may be used to provide data used to control a vehicle.

## Claims

### Claim 1

A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
receiving a dataset comprising a plurality of voxels associated with an environment;
determining, based at least in part on the dataset, ground truth data comprising at least an occupancy status for individual voxels of the plurality of voxels;
determining, based at least in part on the ground truth data, a loss for an individual occupied voxel of the plurality of voxels;
determining, based at least in part on the occupancy status for the individual voxels, a density of proximate occupied voxels for the individual occupied voxel;
determining, based at least in part on the density of the proximate occupied voxels and the loss, an adjusted loss for the individual occupied voxel; and
training a machine-learned (ML) object detection model based at least in part on the adjusted loss.

### Claim 2 (depends on 1)

The system of claim 1, wherein determining the adjusted loss comprises adjusting the loss inversely proportionally to the density of the proximate occupied voxels.

### Claim 3 (depends on 1)

The system of claim 1, wherein training the ML object detection model comprises training the ML object detection model to generate, using an input dataset comprising a plurality of input voxels as input, an object label for individual occupied voxels of the plurality of input voxels as output.

### Claim 4 (depends on 1)

The system of claim 1, wherein training the ML object detection model comprises training the ML object detection model to generate, using an input dataset comprising a plurality of input voxels as input, an occupancy label for individual voxels of the plurality of input voxels as output.

### Claim 5 (depends on 1)

The system of claim 1, wherein the operations further comprise transmitting the ML object detection model to a vehicle configured to traverse a second environment based at least in part on output received from the ML object detection model.

### Claim 6

A method comprising:
receiving a dataset comprising a plurality of voxels associated with an environment and loss values for individual voxels of the plurality of voxels;
determining a loss for an individual occupied voxel of the plurality of voxels;
determining an occupancy status for a subset of the plurality of voxels proximate to the individual occupied voxel;
determining, based at least in part on the occupancy status for individual voxels of the subset of the plurality of voxels, a density of proximate occupied voxels;
determining, based at least in part on the density of the proximate occupied voxels and the loss, an adjusted loss for the individual occupied voxel; and
training a machine-learned (ML) object detection model based at least in part on the adjusted loss.

### Claim 7 (depends on 6)

The method of claim 6, wherein the dataset comprises one or more voxels representing an object protrusion associated with an object in the environment.

### Claim 8 (depends on 6)

The method of claim 6, wherein training the ML object detection model comprises training the ML object detection model to generate, using an input dataset comprising a plurality of input voxels as input, output comprising:
a first object label for a first individual voxel of the plurality of input voxels representing an object, and
a second object label for a second individual voxel of the plurality of input voxels representing an object protrusion associated with the object,
wherein the first object label and the second object label are a same object label.

### Claim 9 (depends on 8)

The method of claim 8, wherein training the ML object detection model further comprises training the ML object detection model to generate a single contour representing the object and the object protrusion.

### Claim 10 (depends on 6)

The method of claim 6, wherein the subset of the plurality of voxels comprises a symmetrical three-dimensional voxel space about the individual occupied voxel.

### Claim 11 (depends on 6)

The method of claim 6, wherein the individual occupied voxel represents at least one of sensor data associated with a space in the environment or data based at least in part on the sensor data.

### Claim 12 (depends on 6)

The method of claim 6, wherein determining the loss comprises:
determining ground truth data associated with the dataset; and
determining the loss for an individual occupied voxel based at least in part on the ground truth data.

### Claim 13 (depends on 6)

The method of claim 6, further comprising:
configuring the ML object detection model at a vehicle computing device;
executing the ML object detection model to generate output; and
controlling a vehicle by the vehicle computing device based at least in part on the output.

### Claim 14 (depends on 6)

The method of claim 6, wherein the adjusted loss is inversely proportional to the density of the proximate occupied voxels.

### Claim 15

One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, perform operations comprising:
receiving a dataset comprising a plurality of data units associated with an environment;
determining a loss for an individual data unit of the plurality of data units;
determining an occupancy status for a subset of the plurality of data units proximate to the individual data unit;
determining, based at least in part on the occupancy status for individual data units of the subset of the plurality of data units, a density of proximate occupied data units;
determining, based at least in part on the density of the proximate occupied data units and the loss, an adjusted loss for the individual data unit; and
training a machine-learned (ML) object detection model based at least in part on the adjusted loss.

### Claim 16 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the individual data unit represents at least one of an occupancy status, a velocity, an acceleration, or a direction.

### Claim 17 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the occupancy status for the subset of the plurality of data units is determined based on sensor data represented in the subset of the plurality of data units.

### Claim 18 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the adjusted loss is inversely proportional to the density of the proximate occupied data units.

### Claim 19 (depends on 15)

The one or more non-transitory computer-readable media of claim 15, wherein the operations further comprise controlling a vehicle based at least in part on output generated by executing the ML object detection model.

### Claim 20 (depends on 19)

The one or more non-transitory computer-readable media of claim 19, wherein the operations further comprise generating a trajectory for controlling the vehicle based at least in part on the output.