Tools for performance testing and/or training autonomous vehicle planners

A non-transitory medium embodying computer-readable instructions for training a target planner to imitate an expert planner for an ego robot, the computer-readable instructions configured, upon execution by one or more processors, to cause operations of:

• applying the target planner in a simulated scenario at incrementing planning steps, in order to compute a series of ego plans that respond to changes in the scenario, the series of ego plans being implemented in the scenario to cause changes in an ego state;
• computing an expert plan for at least one of the planning steps, by applying the expert planner to a scenario state at that planning step;
• using the scenario state and the expert plan as a training example, to train one or more models of the target planner to imitate the expert planner, the training performed with an objective of minimizing deviation between the ego plan computed by the target planner when applied to that scenario state and the expert plan computed by the expert planner when applied to that scenario state, the scenario state including the ego state at said planning step as caused by implementing one or more preceding ego plans of the series of ego plans computed by the target planner;
• and
• generating, based on output of the trained models of the target planner, control signals for controlling an ego robot.

A computer system for evaluating a performance of a target planner for an ego robot in a simulated scenario, the computer system comprising:

• one or more computers configured to implement operations of:
• receiving evaluation data for evaluating performance of the target planner, the evaluation data generated by applying the target planner in a first instance of the scenario from an initial scenario state at first incrementing planning steps, in order to compute a series of ego plans that respond to changes in the first instance of the scenario, a first series of ego plans being implemented in the first instance of the scenario to cause changes in a first ego state, wherein an actual ego trajectory is defined by the changes in the first ego state over a duration of the first instance of the scenario, the evaluation data comprising the initial scenario state and the actual ego trajectory;
• wherein the one or more computers are configured to use the evaluation data to evaluate the target planner by:
• applying a reference planner in a second instance of the scenario from a same initial scenario state at second incrementing planning steps, in order to compute a series of reference plans that respond to changes in the second instance of the scenario, the series of reference plans being implemented in the second instance of the scenario to cause changes in a second ego state, wherein an actual reference trajectory is defined by the changes in the second ego state over a duration of the second instance of the scenario, and
• computing at least one evaluation score for comparing the actual ego trajectory with the actual reference trajectory;
• wherein implementing the series of ego plans comprises, for each ego plan, generating control signals for controlling the ego robot, and
• wherein implementing the series of reference plans comprises, for each reference plan, generating control signals for controlling the ego robot.