Post-processing by lidar system guided by camera information
AssigneeLuminar Technologies, Inc.
InventorRichmond Hicks
Life of this patent
Estimating the term…
Abstract
Post-processing in a lidar system may be guided by camera information as described herein. In one embodiment, a camera system has a camera to capture images of the scene. An image processor is configured to classify an object in the images from the camera. A lidar system generates a point cloud of the scene and a modeling processor is configured to correlate the classified object to a plurality of points of the point cloud and to model the plurality of points as the classified object over time in a 3D model of the scene.
Claims19
A scene modeling system comprising:
- a camera system having a camera to capture images of the scene;
- an image processor configured to classify an object in the images from the camera using the images;
- a lidar system to generate a point cloud of the scene;
- and
- a modeling processor configured to generate a 3D model of the scene using the point cloud by generating a three-dimensional grid of the scene, determining free, occupied, and unknown portions of the grid, configured to correlate the classified object to a plurality of points of the point cloud, configured to assign the classified object to the correlated plurality of points, configured to locate the classified object on the grid, configured to account for the classified object in the occluded portions of the grid, and configured to model the plurality of points as the classified object over time in the 3D model of the scene.
A method of modeling a scene comprising:
- receiving captured images of a scene from a camera;
- classifying an object in the images from the camera using the images;
- receiving a lidar point cloud of the scene;
- generating a 3D model of the scene using the point cloud by generating a three-dimensional grid of the scene, determining free, occupied, and unknown portions of the grid;
- correlating the classified object to a plurality of points of the point cloud;
- assigning the classified object to the correlated plurality of points;
- locating the classified object on the grid;
- accounting for the classified object in the occluded portions of the grid;
- and
- modeling the plurality of points as the classified object over time in the 3D model of the scene.
One or more computer-readable non-transitory storage media embodying software that is operable when executed to perform operations comprising:
- receiving captured images of a scene from a camera;
- classifying an object in the images from the camera using the images;
- receiving a lidar point cloud of the scene;
- generating a 3D model of the scene using the point cloud by generating a three-dimensional grid of the scene, determining free, occupied, and unknown portions of the grid;
- correlating the classified object to a plurality of points of the point cloud;
- assigning the classified object to the correlated plurality of points;
- locating the classified object on the grid;
- accounting for the classified object in the occluded portions of the grid;
- and
- modeling the plurality of points as the classified object over time in the 3D model of the scene.
Dependencies are inferred from the claim text.