Autonomous vehicle validation using real-world adversarial events

A system, comprising:

• at least one processor, and
• at least one non-transitory storage media storing instructions that, when executed by the at least one processor, cause the at least one processor to:
• obtain an organic record of human driver data, the human driver data associated with operation of vehicles by human drivers during at least one period of time;
• extract at least one safety critical scenario from the organic record to create a test data set associated with operation of vehicles by human drivers during safety critical scenarios, wherein predetermined criteria are applied to the organic record to automatically label an event in the organic record as the at least one safety critical scenario;
• simulate operation of a vehicle under evaluation during safety critical scenarios according to the test data set by applying the at least one safety critical scenario to one or more systems of the vehicle under evaluation;
• and
• compare a response of the vehicle under evaluation during the simulated operation to the operation of vehicles by human drivers in the human driver data, wherein the comparison is a homogenous comparison to validate the response of the vehicle under evaluation during the simulated operation according to the test data set, wherein the response of the vehicle is transformed into a safety indicator based on a performance of the vehicle under evaluation.

At least one non-transitory storage media storing instructions that, when executed by at least one processor, cause the at least one processor to:

• obtain an organic record of human driver data, the human driver data associated with operation of vehicles by human drivers during at least one period of time;
• extract at least one safety critical scenario from the organic record to create a test data set associated with operation of vehicles by human drivers during safety critical scenarios, wherein predetermined criteria are applied to the organic record to automatically label an event in the organic record as the at least one safety critical scenario;
• simulate operation of a vehicle under evaluation during safety critical scenarios according to the test data set by applying the at least one safety critical scenario to one or more computer-readable mediums of the vehicle under evaluation;
• and
• compare a response of the vehicle under evaluation during the simulated operation to the operation of vehicles by human drivers in the human driver data, wherein the comparison is a homogenous comparison to validate the response of the vehicle under evaluation during the simulated operation according to the test data set, wherein the vehicle response is transformed into a safety indicator based on a performance of the vehicle under evaluation.