Autonomous vehicle controlled based upon a lidar data segmentation system and an aligned heightmap

A method for controlling operation of an autonomous vehicle (AV), comprising:

• receiving lidar data from a lidar sensor system of the AV, the lidar data based upon output of a lidar sensor included on the AV, the lidar data comprising a plurality of points representative of positions of objects in a driving environment of the AV;
• computing an alignment between the lidar data and a heightmap based upon a first point in the points, the heightmap representative of height of ground cover at a plurality of locations in the driving environment, the first point selected for the computing based upon output of a deep neural network (DNN);
• and
• controlling at least one of an engine of the AV, a braking system of the AV, or a steering system of the AV during operation of the AV in the driving environment based upon the lidar data and the heightmap being aligned to one another.

An autonomous vehicle (AV) comprising:

• a computer-readable storage medium comprising instructions that, when executed by a processor, cause the processor to perform acts comprising:
• receiving lidar data from a lidar sensor system of the AV, the lidar data based upon output of a lidar sensor included on the AV, the lidar data comprising a plurality of points representative of positions of objects in a driving environment of the AV;
• computing, in real time during operation of the AV in the driving environment, an alignment between the lidar data and a heightmap based upon a first point in the points, the heightmap representative of height of ground cover at a plurality of locations in the driving environment, the first point selected for the computing based upon output of a deep neural network (DNN);
• and
• controlling at least one of an engine of the AV, a braking system of the AV, or a steering system of the AV during operation of the AV in the driving environment based upon the lidar data and the heightmap being aligned to one another.