Vehicle navigation based on aligned image and LIDAR information

A non-transitory computer readable medium containing instructions that when executed by at least one processor, cause the at least one processor to perform a method, the method comprising:

• receiving a stream of images captured by a camera onboard a vehicle, wherein the captured images are representative of an environment surrounding the vehicle;
• receiving an output of a LIDAR onboard the vehicle, wherein the output of the LIDAR is representative of a plurality of laser reflections from at least a portion of the environment surrounding the vehicle;
• determining at least one indicator of relative alignment between the output of the LIDAR and at least one image captured by the camera, wherein determining the at least one indicator of relative alignment comprises determining a transform between the at least one image captured by the camera and the output of the LIDAR and accounting for a difference between a scan rate of the LIDAR and a scan rate of the camera, and wherein the at least one indicator of relative alignment is based on aligning, in image space, the output of the LIDAR and the at least one image;
• analyzing the at least one image captured by the camera to identify one or more objects represented in the at least one image;
• attributing, based on the at least one indicator of the relative alignment between the output of the LIDAR and the at least one image captured by the camera, LIDAR reflection information to the one or more objects identified by the analysis of the at least one image captured by the camera;
• and
• using the attributed LIDAR reflection information and the one or more objects identified in the at least one image to determine at least one of an elevation associated with a road in an environment of the vehicle or a speed of the vehicle and to determine a distance between the host vehicle and an area where at least one of the one or more objects contacts the road.

A method of autonomously navigating a host vehicle, the method comprising:

• receiving a stream of images captured by a camera onboard the host vehicle, wherein the captured images are representative of an environment surrounding the host vehicle;
• receiving an output of a LIDAR onboard the host vehicle, wherein the output of the LIDAR is representative of a plurality of laser reflections from at least a portion of the environment surrounding the host vehicle;
• determining at least one indicator of relative alignment between the output of the LIDAR and at least one image captured by the camera, wherein determining the at least one indicator of relative alignment comprises determining a transform between the at least one image captured by the camera and the output of the LIDAR and accounting for a difference between a scan rate of the LIDAR and a scan rate of the camera, and wherein the at least one indicator of relative alignment is based on aligning, in image space, the output of the LIDAR and the at least one image;
• analyzing the at least one image captured by the camera to identify one or more objects represented in the at least one image;
• attributing, based on the at least one indicator of the relative alignment between the output of the LIDAR and the at least one image captured by the camera, LIDAR reflection information to the one or more objects identified by the analysis of the at least one image captured by the camera;
• and
• using the attributed LIDAR reflection information and the one or more objects identified in the at least one image to determine at least one navigational characteristic associated with the host vehicle, the at least one navigational characteristic comprising a distance between the host vehicle and an area where at least one of the one or more objects contacts the road.