Sensor aggregation framework for autonomous driving vehicles

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

• collecting sensor data from one or more sensors mounted on an autonomous driving vehicle (ADV) while the ADV is moving within a region of interest (ROI) that includes a plurality of obstacles, the sensor data including obstacle information of the obstacles and vehicle data of the ADV;
• timestamping each of the vehicle data with a current time at which the vehicle data is captured to generate a plurality of timestamps that correspond to the vehicle data;
• and
• aggregating the obstacle information, the vehicle data, and the corresponding timestamps into training data, wherein the training data is used to train a set of parameters that is subsequently utilized to predict at least in part future obstacle behaviors and vehicle movement of the ADV.

A data processing system, comprising:

• a processor;
• and
• a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including
• collecting sensor data from one or more sensors mounted on an autonomous driving vehicle (ADV) while the ADV is moving within a region of interest (ROI) that includes a plurality of obstacles, the sensor data including obstacle information of the obstacles and vehicle data of the ADV;
• timestamping each of the vehicle data with a current time at which the vehicle data is captured to generate a plurality of timestamps that correspond to the vehicle data;
• and
• aggregating the obstacle information, the vehicle data, and the corresponding timestamps into training data, wherein the training data is used to train a set of parameters that is subsequently utilized to predict at least in part future obstacle behaviors and vehicle movement of the ADV.

A computer-implemented method for operating an autonomous driving vehicle, the method comprising:

• collecting sensor data from one or more sensors mounted on an autonomous driving vehicle (ADV) while the ADV is operating in autonomous mode;
• timestamping each of the sensor data with a current time at which the sensor data is captured to generate a plurality of timestamps that correspond to the sensor data;
• extracting vehicle data of the ADV and obstacle information of perceived obstacles from the sensor data;
• and
• predicting behaviors of the perceived obstacles and vehicle movement of the ADV based on the extracted vehicle data, the extracted obstacle information, the corresponding timestamps, and a set of training parameters that is previously trained to learn historical obstacle behaviors and vehicle movement of the ADV.