Systems and methods for training a driving agent based on real-world driving data

A device, comprising:

• one or more processors configured to:
• receive video data that includes a plurality of video frames and corresponding global positioning system (GPS) data and inertial measurement unit (IMU) data associated with a vehicle;
• align GPS speed, using the IMU data, to compensate for GPS speed delay and to generate modified GPS data,
• wherein the GPS speed and the GPS speed delay are associated with the vehicle;
• remove video frames from the video data to generate modified video data;
• select objects and image regions of video frames of the modified video data;
• determine a current speed and a current turn angle of the vehicle based on the modified GPS data, the IMU data, and the modified video data;
• mask the objects of the video frames of the modified video data to learn first features of the video frames of the modified video data;
• mask the image regions of the video frames of the modified video data to learn second features of the video frames of the modified video data;
• generate a trained neural network model based on the current speed, the current turn angle, the first features, and the second features;
• and
• perform one or more actions based on the trained neural network model.

A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising:

• one or more instructions that, when executed by one or more processors of a device, cause the device to:
• receive video data that includes a plurality of video frames and corresponding global positioning system (GPS) data and inertial measurement unit (IMU) data associated with a vehicle;
• remove video frames from the video data to generate modified video data;
• select objects and image regions of video frames of the modified video data;
• determine a current speed and a current turn angle of the vehicle based on the GPS data, the IMU data, and the modified video data;
• mask the objects of the video frames of the modified video data to learn first features of the video frames of the modified video data;
• mask the image regions of the video frames of the modified video data to learn second features of the video frames of the modified video data;
• generate a trained neural network model based on the current speed, the current turn angle, the first features, and the second features;
• and
• implement the trained neural network model in the vehicle to enable the vehicle to predict a future speed and a future turn angle of the vehicle based on additional video data.