LIDAR-camera fusion where LIDAR and camera validly see different things

A method for determining a navigational action for a host vehicle, the method comprising:

• receiving from a camera onboard the host vehicle at least one captured image representative of an environment of the host vehicle, wherein the camera is positioned at a first location relative to the host vehicle;
• receiving point cloud information from a LIDAR system onboard the host vehicle, wherein the point cloud information is representative of distances to various objects in an environment of the host vehicle, wherein the LIDAR system is positioned at a second location relative to the host vehicle, wherein the second location is different from the first location, and wherein a field of view of the camera at least partially overlaps with a field of view of the LIDAR system to provide a shared field of view region;
• analyzing the at least one captured image and the received point cloud information to detect one or more objects in the shared field of view region, wherein the detected one or more objects are represented in only one of the at least one captured image or the received point cloud information;
• determining whether a vantage point difference between the first location of the camera and the second location of the LIDAR system accounts for the one or more detected objects being represented in only one of the at least one captured image or the received point cloud information;
• if the vantage point difference does not account for the one or more detected objects being represented in only one of the at least one captured image or the received point cloud information, then causing at least one remedial action to be taken;
• and
• if the vantage point difference does account for the one or more detected objects being represented in only one of the at least one captured image or the received point cloud information, then determining at least one navigational action to be taken by the host vehicle based on the one or more detected objects and cause the at least one navigational action to be taken by the host vehicle.

A computer-readable medium storing instructions that when executed by at least one processor, are configured to cause the at least one processor programmed to:

• receive from a camera onboard the host vehicle at least one captured image representative of an environment of the host vehicle, wherein the camera is positioned at a first location relative to the host vehicle;
• receive point cloud information from a LIDAR system onboard the host vehicle, wherein the point cloud information is representative of distances to various objects in an environment of the host vehicle, wherein the LIDAR system is positioned at a second location relative to the host vehicle, wherein the second location is different from the first location, and wherein a field of view of the camera at least partially overlaps with a field of view of the LIDAR system to provide a shared field of view region;
• analyze the at least one captured image and the received point cloud information to detect one or more objects in the shared field of view region, wherein the detected one or more objects are represented in only one of the at least one captured image or the received point cloud information;
• determine whether a vantage point difference between the first location of the camera and the second location of the LIDAR system accounts for the one or more detected objects being represented in only one of the at least one captured image or the received point cloud information;
• if the vantage point difference does not account for the one or more detected objects being represented in only one of the at least one captured image or the received point cloud information, then cause at least one remedial action to be taken;
• and
• if the vantage point difference does account for the one or more detected objects being represented in only one of the at least one captured image or the received point cloud information, then determine at least one navigational action to be taken by the host vehicle based on the one or more detected objects and cause the at least one navigational action to be taken by the host vehicle.