US 10,810,445 B1 · Pipeline with point cloud filtering

Abstract

Techniques are disclosed for filtering point cloud data associated with particulate matter (e.g., gas, exhaust, fog, etc.) which do not impact driving from data used to plan a trajectory and/or a route of a robotic platform. The filtering may be based on determining that a set of points associated with a point cloud represents a navigable space for the robotic platform. The point cloud data may be filtered based on a determination that one or more safety conditions are satisfied. Semantic segmentation may be performed on an image to determine pixel classification probability distributions associated with pixels of the image. Data associated with the set of points may be projected onto the image to identify corresponding pixels. Confidence scores associated with pixel classification probability distributions for the identified pixels may be queried. A classification probability distribution for the set of points may be determined based at least in part on the queried confidence scores.

Claims20

Dependent claims

An autonomous vehicle comprising:

one or more processors;
and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising:
receiving LIDAR data associated with a set of LIDAR points;
receiving image data associated with an image of a portion of an environment associated with the set of LIDAR points;
performing semantic segmentation on the image at multiple scales to determine pixel classifications for pixels of the image, the pixel classifications comprising:
a first confidence score associated with a first scale;
and
a second confidence score associated with a second scale different than the first scale;
upsampling or downsampling the second confidence score to the first scale to generate an upsampled or downsampled second confidence score that is combinable at a same scale with the first confidence score;
combining, as a combined confidence score, the first confidence score and the upsampled or downsampled second confidence score;
and
determining, based at least in part on the combined confidence score, a classification for the set of LIDAR points.

10 more elements

A system comprising:

one or more processors;
and
one or more non-transitory computer-readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
receiving first data from a first sensor, the first data associated with an image of a portion of an environment proximate the system;
determining first confidence scores for pixels in the image associated with one or more classifications;
scaling the image to form a scaled image;
determining second confidence scores for pixels in the scaled image associated with the one or more classifications;
upsampling or downsampling the second confidence scores to generate an upsampled or downsampled second confidence score that is combinable at a same scale with the first confidence scores;
combining the first confidence scores and the upsampled or downsampled second confidence scores to generate a confidence combination;
performing, based at least in part on a set of query pixel coordinates, an interpolation of the confidence combination;
and
determining a pixel classification.

9 more elements

A computer-implemented method comprising:

obtaining an image from a camera, the image associated with a portion of an environment;
determining a first confidence output and a second confidence output, wherein the first confidence output is associated with a first scale and the second confidence output is associated with a second scale different than the first scale, the first confidence output comprising a set of confidences that pixels of the image are associated with one or more classifications and the second confidence output comprising a set of confidences that pixels of the image are associated with the one or more classifications;
upsampling or downsampling, based at least in part on an interpolation, the second confidence output to generate an upsampled or downsampled second confidence output that is combinable at a same scale with the first confidence output;
combining the first confidence output and the upsampled or downsampled second confidence output as a confidence combination;
and
determining, based at least in part on a pixel coordinate and the confidence combination, a classification associated with the pixel coordinate.

3 more elements

Dependencies are inferred from the claim text.

Pipeline with point cloud filtering

Life of this patent

Pipeline with point cloud filtering

Life of this patent

Abstract

Claims20