US 11,636,685 B1Granted

Multi-resolution top-down segmentation

AssigneeZoox, Inc.

InventorsStanley L. Volta, Allan Zelener

Life of this patent

Estimating the term…

Loading patent…

Abstract

Techniques for segmenting sensor data are discussed herein. Data can be represented in individual levels in a multi-resolution voxel space. A first level can correspond to a first region of an environment and a second level can correspond to a second region of an environment that is a subset of the first region. In some examples, the levels can comprise a same number of voxels, such that the first level covers a large, low-resolution region, while the second level covers a smaller, higher-resolution region, though more levels are contemplated. Operations may include analyzing sensor data represented in the voxel space from a perspective, such as a top-down perspective. From this perspective, techniques may generate masks that represent objects in the voxel space. Additionally, techniques may generate segmentation data to verify and/or generate the masks, or otherwise cluster the sensor data.

Claims20

Dependent claims

A system comprising:

one or more processors;
and
one or more non-transitory computer-readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
receiving sensor data from a sensor associated with an autonomous vehicle;
associating a first portion of the sensor data with a first voxel space comprising a first range;
associating a second portion of the sensor data with a second voxel space comprising a second range, the first range being less than the second range, and the second range containing the first range;
determining first image data representing a first top-down view of the first voxel space, wherein a first pixel of the first image data indicates an occupancy of a first column of the first voxel space;
determining second image data representing a second top-down view of the second voxel space, where a second pixel of the second image data indicates an occupancy of a second column of the second voxel space;
inputting the first image data and the second image data into a machine learned (ML) model;
receiving, from the ML model, a segmentation mask associated with an object;
clustering, based at least in part on the segmentation mask, a third portion of the sensor data to determine the object;
and
controlling the autonomous vehicle based at least in part on the object.

10 more elements

One or more non-transitory computer-readable media storing instructions executable by a processor, wherein the instructions, when executed, cause the processor to perform operations comprising:

receiving sensor data from a sensor associated with an autonomous vehicle;
associating the sensor data with a first voxel space and a second voxel space, the first voxel space being different from the second voxel space;
determining first image data representing a first top-down view of the first voxel space, wherein a first pixel of the first image data indicates an occupancy of a first column of the first voxel space;
determining second image data representing a second top-down view of the second voxel space, wherein a second pixel of the second image data indicates an occupancy of a second column of the second voxel space;
inputting the first image data and the second image data into a machine learned (ML) model;
clustering, based at least in part on output data received from the ML model, a third portion of the sensor data to determine an object;
and
controlling the autonomous vehicle based at least in part on the object.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/127,196Filed 18 Dec 2020
Granted: 25 Apr 2023
Term ends: Not estimatedThe source records don’t support an estimate.

Classification20 codes

G05D 1/024
Using obstacle or wall sensors: in combination with a laser
G06V 20/56
Context or environment of the image: exterior to a vehicle by using sensors mounted on the vehicle
G05D 1/0088
Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots: characterized by the…
G05D 1/0251
Using a video camera in combination with image processing means: extracting 3D information from a plurality of images taken from different…