Training a neural network to generate a dense depth map from image and lidar data

A device for training a neural network, the device comprising:

• a memory configured to store a neural network model for the neural network;
• and
• a processing system comprising one or more processors implemented in circuitry, the processing system being configured to:
• extract image features from an image of an area, the image features representing objects in the area;
• extract point cloud features from a point cloud representation of the area, the point cloud features representing the objects in the area;
• add Gaussian noise to a ground truth depth map for the area to generate a noisy ground truth depth map, the ground truth depth map representing accurate positions of the objects in the area;
• and
• train the neural network using the image features, the point cloud features, and the noisy ground truth depth map to generate a depth map.

A device for processing image data using a neural network, the device comprising:

• a memory configured to store a neural network model for the neural network, the neural network having been trained using Gaussian noise added to a ground truth depth map for a first area, first image features extracted from a first image of the first area, and point cloud features extracted from a first point cloud representation of the first area, the first image features representing first objects in the first area, and the first point cloud features representing the first objects in the first area;
• and
• a processing system comprising one or more processors implemented in circuitry, the processing system being configured to:
• extract second image features from a second image of a second area, the second image features representing second objects in the second area;
• extract second point cloud features from a second point cloud representation of the second area, the second point cloud features representing the second objects in the second area;
• and
• provide the second image features and the second point cloud features to the neural network to generate a depth map for the second area.