Systems and methods for determining road object importance based on forward facing and driver facing video data

A device, comprising:

• one or more processors configured to:
• receive driver facing video data associated with a driver of a vehicle and forward facing video data associated with the vehicle;
• process the driver facing video data, with a face model, to identify driver head orientation and driver gaze;
• utilize a face mesh model, an eye key points extraction model, and a perspective-n-points estimation model to generate a first transformation matrix mapping the driver facing video data, the driver head orientation, and the driver gaze;
• generate a second transformation matrix mapping the driver facing video data and the forward facing video data;
• utilize the first transformation matrix and the second transformation matrix to estimate forward facing image coordinates observed by the driver;
• aggregate the forward facing image coordinates to generate aggregated coordinates;
• generate heat maps based on the aggregated coordinates,
• wherein the heat maps are associated with regions of the forward facing video data that are relevant to driving;
• train a machine learning model, with the heat maps, to generate a trained machine learning model;
• and
• perform one or more actions based on the trained machine learning model.

A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising:

• one or more instructions that, when executed by one or more processors of a device, cause the device to:
• receive forward facing video data associated with a vehicle;
• process the forward facing video data, with a trained machine learning model, to predict one or more regions in the forward facing video data that are semantically relevant to driving the vehicle,
• wherein the trained machine learning model is trained based on historical driver facing video data associated with a plurality of drivers of a plurality of vehicles and historical forward facing video data associated with the plurality of vehicles;
• and
• perform one or more actions based on the one or more regions in the forward facing video data that are semantically relevant to driving the vehicle.