Vehicular multi-sensor system using a camera and lidar sensor to detect objects

A vehicular multi-sensor system, said vehicular multi-sensor system comprising:

• a plurality of sensors disposed at a vehicle equipped with said vehicular multi-sensor system;
• wherein said plurality of sensors at least comprises (i) a forward-viewing camera, said forward-viewing camera disposed behind a windshield of the equipped vehicle and viewing forward of the equipped vehicle through the windshield, (ii) a forward-sensing 3D point-cloud LIDAR sensor having a field of sensing forward of the equipped vehicle and (iii) a forward-sensing night vision camera;
• wherein said forward-viewing camera is operable to capture image data;
• wherein said forward-sensing 3D point-cloud LIDAR sensor is operable to capture 3D point-cloud LIDAR data;
• wherein said forward-sensing night vision camera is operable to capture image data;
• wherein said forward-viewing camera comprises a photosensor array having at least one million photosensor elements arranged in a two-dimensional array of columns and rows;
• wherein said forward-viewing camera views (i) a traffic lane of a multi-lane road being traveled along by the equipped vehicle and (ii) another traffic lane of the multi-lane road;
• wherein the field of sensing of said forward-sensing 3D point-cloud LIDAR sensor at least encompasses the other traffic lane of the multi-lane road;
• an electronic control unit (ECU) comprising electronic circuitry;
• wherein image data captured by said forward-viewing camera is transferred from said forward-viewing camera to the ECU and is processed at the ECU;
• wherein 3D point-cloud LIDAR data captured by said forward-sensing 3D point-cloud LIDAR sensor is transferred from said forward-sensing 3D point-cloud LIDAR sensor to the ECU and is processed at the ECU;
• wherein image data captured by said forward-sensing night vision camera is transferred from said forward-sensing night vision camera to the ECU and is processed at the ECU;
• wherein, responsive at least in part to processing at the ECU of 3D point-cloud LIDAR data captured by said forward-sensing 3D point-cloud LIDAR sensor and of at least one selected from the group consisting of (i) image data captured by said forward-viewing camera and (ii) image data captured by said forward-sensing night vision camera, a traffic participant present forward of the equipped vehicle on the other traffic lane of the multi-lane road is detected;
• wherein the traffic participant present forward of the equipped vehicle on the other traffic lane of the multi-lane road comprises a pedestrian;
• wherein image data captured by said forward-viewing camera is processed at the ECU for a headlamp control system of the equipped vehicle and for at least one selected from the group consisting of (i) a lane departure warning system of the equipped vehicle, (ii) a traffic sign recognition system of the equipped vehicle and (iii) an adaptive cruise control system of the equipped vehicle;
• and
• wherein the headlamp control system of the equipped vehicle adjusts a headlamp of the equipped vehicle based at least in part on detection of the traffic participant responsive at least in part to processing at the ECU of 3D point-cloud LIDAR data captured by said forward-sensing 3D point-cloud LIDAR sensor.

A vehicular multi-sensor system, said vehicular multi-sensor system comprising:

• a plurality of sensors disposed at a vehicle equipped with said vehicular multi-sensor system;
• wherein said plurality of sensors at least comprises (i) a forward-viewing camera, said forward-viewing camera disposed behind a windshield of the equipped vehicle and viewing forward of the equipped vehicle through the windshield and (ii) a forward-sensing 3D point-cloud LIDAR sensor having a field of sensing forward of the equipped vehicle;
• wherein said forward-viewing camera is operable to capture image data;
• wherein said forward-sensing 3D point-cloud LIDAR sensor is operable to capture 3D point-cloud LIDAR data;
• wherein said forward-viewing camera comprises a photosensor array having at least one million photosensor elements arranged in a two-dimensional array of columns and rows;
• wherein said forward-viewing camera views (i) a traffic lane of a multi-lane road being traveled along by the equipped vehicle and (ii) another traffic lane of the multi-lane road;
• wherein the field of sensing of said forward-sensing 3D point-cloud LIDAR sensor at least encompasses the other traffic lane of the multi-lane road;
• an electronic control unit (ECU) comprising electronic circuitry;
• wherein image data captured by said forward-viewing camera is transferred from said forward-viewing camera to the ECU and is processed at the ECU;
• wherein 3D point-cloud LIDAR data captured by said forward-sensing 3D point-cloud LIDAR sensor is transferred from said forward-sensing 3D point-cloud LIDAR sensor to the ECU and is processed at the ECU;
• wherein, responsive at least in part to processing at the ECU (i) of image data captured by said forward-viewing camera and (ii) of 3D point-cloud LIDAR data captured by said forward-sensing 3D point-cloud LIDAR sensor, a traffic participant present forward of the equipped vehicle on the other traffic lane of the multi-lane road is detected;
• wherein the traffic participant present forward of the equipped vehicle on the other traffic lane of the multi-lane road comprises another vehicle present forward of the equipped vehicle;
• wherein image data captured by said forward-viewing camera is processed at the ECU for a headlamp control system of the equipped vehicle and for at least one selected from the group consisting of (i) a lane departure warning system of the equipped vehicle, (ii) a traffic sign recognition system of the equipped vehicle and (iii) an adaptive cruise control system of the equipped vehicle;
• and
• wherein the headlamp control system of the equipped vehicle adjusts a headlamp of the equipped vehicle based at least in part on detection of the traffic participant responsive at least in part to processing at the ECU of 3D point-cloud LIDAR data captured by said forward-sensing 3D point-cloud LIDAR sensor.