Evaluating driving data with a modular and configurable evaluation framework

A system comprising:

• one or more processors and one or more computer-readable non-transitory storage media, the one or more computer-readable non-transitory storage media comprising instructions operable when executed by the one or more processors to cause the system to perform operations comprising:
• receiving a request for evaluating driving data included in a data log acquired from sensors of a vehicle;
• instantiating, from an evaluation configuration file associated with the request, one or more metric calculators configured to generate one or more output metrics from the driving data, wherein the evaluation configuration file specifies a validation configuration configured to validate the one or more output metrics,
• wherein instantiating includes modifying generic code snippets into modified snippets according to the metric calculation configuration and compiling and executing, using an electronic processor, the modified snippets to implement the one or more metric calculators by integrating information from the evaluation configuration file to adapt the generic code snippets;
• determining particular driving data from the driving data included in the data log based on the one or more metric calculators;
• and
• generating the output metrics for the particular driving data by using the one or more metric calculators to calculate attributes about control of the vehicle, including dynamics of the vehicle and relationships to objects in a surrounding environment, and wherein the validation configuration specifies limits and conditions on the dynamics and the relationships to determine when control of the vehicle is satisfactory.

One or more computer-readable non-transitory storage media including instructions that, when executed by one or more processors, are configured to cause the one or more processors to perform operations comprising:

• receiving a request for evaluating driving data included in a data log acquired from sensors of a vehicle;
• instantiating, from an evaluation configuration file associated with the request, one or more metric calculators configured to generate one or more output metrics from the driving data, wherein the evaluation configuration file specifies a validation configuration to validate the output metrics and a metric calculation configuration specifying the metric calculators,
• wherein instantiating includes modifying generic code snippets into modified snippets according to the metric calculation configuration and compiling and executing, using an electronic processor, the modified snippets to implement the metric calculators by integrating information from the evaluation configuration file to adapt the generic code snippets;
• determining particular driving data from the driving data included in the data log based on the one or more metric calculators;
• and
• generating the output metrics for the particular driving data by using the metric calculators to calculate attributes about control of the vehicle, including dynamics of the vehicle and relationships to objects in a surrounding environment, and wherein the validation configuration specifies limits and conditions on the dynamics and the relationships to determine when control of the vehicle is satisfactory.