Systems and methods for generating synthetic sensor data via machine learning

One or more non-transitory computer-readable media storing:

• one or more machine-learned models, wherein at least one of the one or more machine-learned models was tested using synthetic light detection and ranging (LiDAR) data, wherein the synthetic LiDAR data was generated by:
• using a physics-based simulation engine to obtain an initial synthetic point cloud,
• generating, by a machine-learned model, a value corresponding to a probability that a real LiDAR point cloud would have an error at a point in the initial synthetic point cloud, and
• generating, based on a determination of whether, based on the value, to include the error in the synthetic LIDAR data, the synthetic LIDAR data to contain the error at the point;
• and
• instructions that are executable by one or more processors to cause an autonomous vehicle control system to perform operations, the operations comprising:
• obtaining LIDAR data descriptive of an environment of the autonomous vehicle;
• determining, using the one or more machine-learned models and based on the LiDAR data, a motion plan for controlling the autonomous vehicle;
• and
• controlling the autonomous vehicle according to the motion plan.

A method, comprising:

• obtaining LiDAR data descriptive of an environment of an autonomous vehicle;
• determining, using one or more machine-learned models and based on the LiDAR data, a motion plan for controlling the autonomous vehicle, wherein at least one of the one or more machine-learned models was tested using synthetic light detection and ranging (LiDAR) data, wherein the synthetic LiDAR data was generated by:
• using a physics-based simulation engine to obtain an initial synthetic point cloud,
• generating, by a machine-learned model, a value corresponding to a probability that a real LiDAR point cloud would have an error at a point in the initial synthetic point cloud,
• generating, based on a determination of whether, based on the value, to include the error in the synthetic LIDAR data, the synthetic LIDAR data to contain the error at the point;
• and
• controlling the autonomous vehicle according to the motion plan.

An autonomous vehicle control system for controlling an autonomous vehicle, the autonomous vehicle control system comprising:

• one or more processors;
• and
• one or more non-transitory computer-readable media storing:
• one or more machine-learned models, wherein at least one of the one or more machine-learned models was tested using synthetic light detection and ranging (LiDAR) data, wherein the synthetic LiDAR data was generated by:
• using a physics-based simulation engine to obtain an initial synthetic point cloud, and
• using a machine-learned model to add realistic errors to the initial synthetic point cloud to obtain the synthetic LiDAR data;
• wherein the at least one of the one or more machine-learned models was tested by processing the synthetic LiDAR data using the at least one of the one or more machine-learned models to test a performance of the at least one of the one or more machine-learned models in a test environment described by the synthetic LiDAR data;
• and
• instructions that are executable by the one or more processors to cause the autonomous vehicle control system to perform operations, the operations comprising:
• obtaining LiDAR data descriptive of an environment of the autonomous vehicle;
• determining, using the one or more machine-learned models and based on the LiDAR data, a motion plan for controlling the autonomous vehicle;
• and
• controlling the autonomous vehicle according to the motion plan.