US 10,509,947 B1Granted

Converting multi-dimensional data for image analysis

AssigneeZoox, Inc.

InventorsBertrand Robert Douillard, Subhasis Das, Zeng Wang, Dragomir Dimitrov Anguelov

Life of this patent

Estimating the term…

Loading patent…

Abstract

Multi-dimensional data can be mapped to a projection shape and converted for image analysis. In some examples, the multi-dimensional data may include data captured by a LIDAR system for use in conjunction with a perception system for an autonomous vehicle. Converting operations can include converting three-dimensional LIDAR data to multi-channel two-dimensional data. Data points of the multi-dimensional data can be mapped to a projection shape, such as a sphere. Characteristics of the projection shape may include a shape, a field of view, a resolution, and a projection type. After data is mapped to the projection shape, the projection shape can be converted to a multi-channel, two-dimensional image. Image segmentation and classification may be performed on the two-dimensional data. Further, segmentation information may be used to segment the three-dimensional LIDAR data, while a rendering plane may be positioned relative to the segmented data to perform classification on a per-object basis.

Claims20

Dependent claims

A system comprising:

one or more processors;
and
one or more non-transitory computer readable storage media communicatively coupled to the one or more processors and storing instructions that are executable by the one or more processors to:
capture a three-dimensional dataset using at least one LIDAR sensor installed on an autonomous vehicle;
determine a projection shape for mapping at least a portion of the three-dimensional dataset to the projection shape, the projection shape having at least a shape comprising one or more of a sphere or a cylinder being oriented based at least in part on a position of the at least one LIDAR sensor, the shape being discretized into a number of cells, the number of cells representing a resolution of the shape;
project, as projected data, the at least the portion of the three-dimensional dataset to one or more cells associated with the projection shape, wherein data associated with a data point of the at least the portion of the three-dimensional dataset is stored in association with a corresponding cell of the projection shape;
convert the projected data to a multi-channel two-dimensional image, wherein the multi-channel two-dimensional image comprises at least a first channel storing first information associated with a first coordinate of the data point and a second channel storing second information associated with a second coordinate of the data point;
and
input the multi-channel two-dimensional image into at least one segmenter and at least one classifier to generate segmentation information and classification information.

6 more elements

A method comprising:

receiving a dataset, data in the dataset having multiple dimensions, the dataset comprising a data point captured by a sensor;
associating at least a portion of the dataset with a projection shape to obtain projected data, wherein the projection shape comprises a shape comprising one or more of a sphere or a cylinder oriented based at least in part on a position of the sensor;
converting the projected data to a multi-channel two-dimensional image, wherein the multi-channel two-dimensional image comprises a first channel storing first information associated with a first coordinate of the data point and a second channel storing second information associated with a second coordinate of the data point;
and
performing segmentation on the multi-channel two-dimensional image to generate segmentation information.

2 more elements

A system comprising:

one or more processors;
and
one or more non-transitory computer readable storage media communicatively coupled to the one or more processors and storing instructions that are executable by the one or more processors to:
receive a LIDAR dataset, the LIDAR dataset having multiple dimensions and comprising a data point captured by a sensor;
associate at least a portion of the LIDAR dataset with a projection shape to obtain projected data, wherein the projection shape comprises a shape comprising one or more of a sphere or a cylinder oriented based at least in part on a position of the sensor;
convert the projected data to a multi-channel two-dimensional image, wherein the multi-channel two-dimensional image comprises a first channel storing first information associated with a first coordinate of the data point and a second channel storing second information associated with a second coordinate of the data point;
and
input the multi-channel two-dimensional image to at least one convolutional neural network to generate segmentation information.

5 more elements

Dependencies are inferred from the claim text.