# US 12,762,839 B2 · Method, apparatus and computer-readable storage medium for predicting target object

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,762,839 B2
- Document ID: US12762839B2
- Status: Granted
- Kind: B2
- Application number: 18/779,028
- Filed: 2024-07-21
- Published: 2025-01-30
- Granted: 2026-10-13
- Earliest claimed: 2023-07-25
- Applicants: Beijing Tusen Zhitu Technology Co., LTD.
- Inventors: Lue Fan; Feng Wang; Naiyan Wang
- Web page: https://patentgrep.com/patent/US12762839B2

## CPC

- G06V 10/764: using classification, e.g. of video objects
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes}
- G06V 2201/07: Target detection
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present disclosure provides a method for predicting a target object, including performing a first voxelization on a plurality of first points included in a point cloud to obtain a plurality of first voxels; predicting a plurality of second points from the plurality of first voxels, wherein each of the plurality of second points represents a prediction of a center of the target object by a first point; performing a second voxelization on a plurality of first points and a plurality of second points to obtain one or more second voxels and one or more third voxels; and predicting the target object from the one or more third voxels. In addition, the present disclosure also provides an apparatus and non-transitory computer-readable storage medium that can perform the methods described above.

## Claims

### Claim 1

A method for predicting a target object, comprising:
performing a first voxelization on a plurality of first points included in a point cloud to obtain a plurality of first voxels;
predicting, based on the plurality of first voxels, a plurality of second points, wherein each of the plurality of second points represents a prediction of a center of the target object based on the plurality of first points;
performing a second voxelization on the plurality of first points and the plurality of second points to obtain one or more second voxels and one or more third voxels; and
predicting, based on the one or more third voxels, the target object,
wherein the one or more second voxels correspond to the plurality of first points and the one or more third voxels correspond to the plurality of second points.

### Claim 2 (depends on 1)

The method according to claim 1, wherein predicting the plurality of second points comprises:
extracting, by a sparse voxel encoder, a plurality of first voxel features of the plurality of first voxels;
obtaining, based on the plurality of first voxel features, a plurality of point data features of the plurality of first points; and
predicting, based on the plurality of point data features, the plurality of second points.

### Claim 3 (depends on 2)

The method according to claim 2, wherein obtaining, based on the plurality of first voxel features, the plurality of point data features for the plurality of first points comprises:
for each of the plurality of first voxels:
mapping a first voxel feature of the first voxel to one or more first points comprised by the first voxel to obtain one or more initial point data features;
calculating one or more geometric features between the one or more first points and the corresponding first voxel; and
generating, based on the one or more initial point data features and the one or more geometric features, one or more point data features of the one or more first points of the first voxel.

### Claim 4 (depends on 2)

The method according to claim 2, wherein predicting, based on the plurality of point data features, the plurality of second points comprises:
predicting, based on a classifier and the plurality of point data features, a plurality of offsets between the plurality of first points and a center of the target object respectively; and
obtaining, based on the plurality of first points and the plurality of offsets, the plurality of second points.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the one or more third voxels are a plurality of third voxels, predicting, based on the one or more third voxels, the target object comprises:
extracting a plurality of third voxel features of the plurality of third voxels;
mixing the plurality of third voxel features to obtain a plurality of first mixed voxel features, wherein each of the plurality of third voxels corresponds to one of the plurality of first mixed voxel features; and
predicting, based on the plurality of first mixed voxel features, the target object.

### Claim 6 (depends on 1)

The method according to claim 1, wherein predicting, based on the one or more third voxels, the target object comprises:
predicting, based on the one or more second voxels and the one or more third voxels, the target object.

### Claim 7 (depends on 6)

The method according to claim 6, wherein predicting, based on the one or more second voxels and the one or more third voxels, the target object comprises:
extracting one or more second voxel features of the one or more second voxels;
extracting one or more third voxel features of the one or more third voxels;
mixing the one or more second voxel features and the one or more third voxel features to obtain a plurality of second mixed voxel features using a mixing module, preferably a mixing module comprising a convolution network model, wherein each of the one or more second voxels corresponds to one of the plurality of second mixed voxel features and each of the one or more third voxels corresponds to one of the plurality of second mixed voxel features; and
predicting, based on the plurality of second mixed voxel features, the target object.

### Claim 8 (depends on 7)

The method according to claim 7, wherein predicting, based on the plurality of second mixture voxel features, the target object comprises:
predicting, by a prediction module, the target object based on the plurality of second mixture voxel features corresponding to the one or more third voxels to generate target prediction markers corresponding to the third voxels.

### Claim 9 (depends on 1)

The method according to claim 1, wherein a size of a voxel obtained by the second voxelization is larger than a size of a voxel obtained by the first voxelization; and
each of the plurality of first voxels comprises a quantity of the first points greater than or equal to a first predetermined value and each of the one or more third voxels comprises a quantity of the second points greater than or equal to a second predetermined value.

### Claim 10 (depends on 1)

The method according to claim 1, wherein predicting, based on the one or more third voxels, the target object comprises:
inputting the one or more third voxels into a neural network to output a target prediction marker through a prediction model of the neural network, the target prediction marker representing a prediction of the target object; and
the prediction method for the target object further comprises:
obtaining a sample target prediction marker which represents a real prediction of the target object; wherein for each of the one or more third voxels:
calculating a centroid position representing one or more second points corresponding to the third voxel; and
training, according to the centroid position and the relationship of the sample target prediction marker, the neural network with the sample target prediction marker.

### Claim 11 (depends on 10)

The method according to claim 10, wherein the one or more second points corresponding to the third voxels comprise one or more foreground points and one or more non-foreground points; and calculating the centroid position representing the one or more second points corresponding to the third voxels comprises:
performing weighted averaging on the positions of the one or more foreground points with a first weight and the positions of the one or more non-foreground points with a second weight, to obtain a centroid position representing one or more second points corresponding to the third voxel, wherein the first weight is greater than the second weight.

### Claim 12 (depends on 10)

The method according to claim 10, wherein the sample target prediction marker is a sample prediction box; and training, according to the centroid position and the relationship of the sample target prediction marker, the neural network with the sample target prediction marker comprises:
training, in response to the centroid position falling in the sample prediction box, the neural network with the sample target prediction marker.

### Claim 13 (depends on 12)

The method according to claim 12, wherein training, in response to that the centroid position falls within the sample prediction box, the neural network with the sample target prediction marker comprises:
comparing the target prediction marker with the sample target prediction marker to obtain a comparison result; and
updating, by the neural network, the prediction model based on the comparison result to output a new target prediction marker, wherein the updated prediction model is used to make prediction of the target object represented by the new target prediction marker closer to the real prediction of the target object.

### Claim 14

An apparatus for predicting a target object, comprising:
a memory storing a computer program; and
a processor coupled to the memory, wherein the processor, when executing the computer program, causes the processor to implement a method comprising:
performing a first voxelization on a plurality of first points included in a point cloud to obtain a plurality of first voxels;
predicting, based on the plurality of first voxels, a plurality of second points, wherein each of the plurality of second points represents a prediction of a center of the target object by a first point;
performing a second voxelization on the plurality of first points and the plurality of second points to obtain one or more second voxels and one or more third voxels; and
predicting, based on the one or more third voxels, the target object,
wherein the one or more second voxels correspond to the plurality of first points and the one or more third voxels correspond to the plurality of second points.

### Claim 15 (depends on 14)

The apparatus according to claim 14, wherein predicting the plurality of second points comprises:
extracting, by a sparse voxel encoder, a plurality of first voxel features of the plurality of first voxels;
obtaining, based on the plurality of first voxel features, a plurality of point data features of the plurality of first points; and
predicting, based on the plurality of point data features, the plurality of second points.

### Claim 16 (depends on 15)

The apparatus according to claim 15, wherein
obtaining, based on the plurality of first voxel features, the plurality of point data features for the plurality of first points comprises:
for each of the plurality of first voxels:
mapping a first voxel feature of the first voxel to one or more first points comprised by the first voxel to obtain one or more initial point data features;
calculating one or more geometric features between the one or more first points and the corresponding first voxels; and
generating, based on the one or more initial point data features and the one or more geometric features, one or more point data features of the one or more first points of the first voxel; and
predicting, based on the plurality of point data features, the plurality of second points comprises:
predicting, based on a classifier and the plurality of point data features, a plurality of offsets between the plurality of first points and a center of the target object respectively; and
obtaining, based on the plurality of first points and the plurality of offsets, the plurality of second points.

### Claim 17 (depends on 14)

The apparatus according to claim 14, wherein predicting, based on the one or more third voxels, the target object comprises:
extracting one or more second voxel features of the one or more second voxels;
extracting one or more third voxel features of the one or more third voxels;
mixing the one or more second voxel features and the one or more third voxel features to obtain a plurality of second mixed voxel features using a mixing module, preferably a mixing module comprising a convolution network model, wherein each of the one or more second voxels corresponds to one of the plurality of second mixed voxel features and each of the one or more third voxels corresponds to one of the plurality of second mixed voxel features; and
predicting, by a prediction module, the target object based on the plurality of second mixture voxel features corresponding to the one or more third voxels to generate target prediction markers corresponding to the third voxels.

### Claim 18

A non-transitory computer-readable storage medium, wherein the computer-readable storage medium stores a computer program which when executed by a processor implements a method for predicting a target object, the method comprising:
performing a first voxelization on a plurality of first points included in a point cloud to obtain a plurality of first voxels;
predicting, based on the plurality of first voxels, a plurality of second points, wherein each of the plurality of second points represents a prediction of a center of the target object by a first point;
performing a second voxelization on the plurality of first points and the plurality of second points to obtain one or more second voxels and one or more third voxels; and
predicting, based on the one or more third voxels, the target object,
wherein the one or more second voxels correspond to the plurality of first points and the one or more third voxels correspond to the plurality of second points.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium according to claim 18,
wherein predicting, based on the one or more third voxels, the target object comprises:
inputting the one or more third voxels into a neural network to output a target prediction marker through a prediction model of the neural network, the target prediction marker representing a prediction of the target object; and
the prediction method for the target object further comprises:
obtaining a sample target prediction marker which represents a real prediction of the target object; wherein for each of the one or more third voxels:
calculating a centroid position representing one or more second points corresponding to the third voxel; and
training, according to the centroid position and the relationship of the sample target prediction marker, the neural network with the sample target prediction marker.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium according to claim 19, wherein the one or more second points corresponding to the third voxels comprise one or more foreground points and one or more non-foreground points, and the sample target prediction marker is a sample prediction box;
calculating the centroid position representing the one or more second points corresponding to the third voxels comprises:
performing weighted averaging on the positions of the one or more foreground points with a first weight and the positions of the one or more non-foreground points with a second weight, to obtain a centroid position representing one or more second points corresponding to the third voxel, wherein the first weight is greater than the second weight; and
training, according to the centroid position and the relationship of the sample target prediction marker, the neural network with the sample target prediction marker comprises:
in response to the centroid position falling in the sample prediction box,
comparing the target prediction marker with the sample target prediction marker to obtain a comparison result; and
updating, by the neural network, the prediction model based on the comparison result to output a new target prediction marker, wherein the updated prediction model is used to make prediction of the target object represented by the new target prediction marker closer to the real prediction of the target object.