Spatial aware object detection by LIDAR and camera fusion based super-resolution
AssigneeContinental Automotive GmbH
InventorRobert-Zsolt Kabai
Life of this patent
Estimating the term…
Abstract
A method for training an artificial intelligence module for an advanced driver assistance system for obtaining a depth model of a scene uses data of a camera image and a depth image of a LIDAR sensor as input, and is trained for outputting the depth model of the scene. The depth model has a higher resolution than the depth image of the LIDAR sensor. The method of training the artificial intelligence module includes acquiring depth measurement data of the scene, which are obtained by combining images of a plurality of other LIDAR sensors, and which are used as a ground truth for the depth model of the scene.
Claims11
A method for training an artificial intelligence module of an advanced driver assistance system for obtaining a depth model of a scene, the method comprising:
- acquiring multiple sets of training data, each set of training data among the multiple sets of training data including a camera image of the scene taken by a camera of a vehicle, a depth image of the scene at a first resolution taken by a LIDAR sensor of the vehicle, the depth image comprising data indicating a distance of each pixel of the depth image from the LIDAR sensor, and depth measurement data of the scene at a second resolution that is higher than the first resolution taken by a plurality of other LIDAR sensors, the depth measurement data comprising data indicating a distance of each pixel of images of the plurality of other LIDAR sensors from the plurality of other LIDAR sensors;
- combining the images of the plurality of other LIDAR sensors to obtain the depth measurement data of the scene from a perspective of the LIDAR sensor of the vehicle, wherein a resolution of the depth measurement data of the scene is higher than a resolution of the images of the plurality of other LIDAR sensors;
- and
- training the artificial intelligence module of the advanced driver assistance system with the multiple sets of training data for obtaining the depth model of the scene with the second resolution if provided with the camera image and the depth image with the first resolution as inputs to the artificial intelligence model.
An advanced driver assistance system, comprising:
- a camera configured to capture an image of the scene;
- a LIDAR sensor configured to capture a depth image of the scene at a first resolution;
- and
- a processor configured to train an artificial intelligence module for obtaining a depth model of the scene by acquiring multiple sets of training data, each set of training data among the multiple sets of training data including a camera image of the scene taken by the camera, a depth image of the scene taken by the LIDAR sensor, the depth image comprising data indicating a distance of each pixel of the depth image from the LIDAR sensor, and depth measurement data of the scene at a second resolution that is higher than the first resolution taken by a plurality of other LIDAR sensors, the depth measurement data comprising data indicating a distance of each pixel of images of the plurality of other LIDAR sensors from the plurality of other LIDAR sensors;
- combine the images of the plurality of other LIDAR sensors to obtain the depth measurement data of the scene from a perspective of the LIDAR sensor of the vehicle, wherein a resolution of the depth measurement data of the scene is higher than a resolution of the images of the plurality of other LIDAR sensors;
- and
- wherein the artificial intelligence module is configured to output the depth model of the scene with the second resolution if provided with the camera image and the depth image with the first resolution as inputs to the artificial intelligence model.
Dependencies are inferred from the claim text.