Systems and methods for autonomous vehicle performance evaluation

A system comprising:

• at least one processor;
• and
• a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
• receiving transportation information associated with a transportation request, the transportation information comprising a pick up location and a drop off location;
• determining a first route associated with the transportation request and a non-autonomous vehicle;
• determining a second route associated with the transportation request and an autonomous vehicle based on an operating design domain (ODD) associated with one or more autonomous vehicles in a fleet of vehicles;
• categorizing disengagements in scenarios associated with the ODD into at least an adverse outcome category and a no adverse outcome category based on expected outcomes determined by simulation of the disengagements;
• determining at least one performance metric associated with the second route based on a weighted average of performance metric values associated with road segments of the second route, wherein the performance metric values are based on the expected outcomes of the disengagements in the adverse outcome category, wherein at least one of the road segments has a default performance metric value determined based on other road segments within a threshold distance of the at least one of the road segments, and wherein the performance metric values are weighted based on frequency by which the road segments were traversed;
• selecting the second route based at least in part on the at least one performance metric and a comparison of the first route and the second route;
• assigning an autonomous vehicle from the fleet of vehicles to the transportation request based on selection of the second route;
• and
• dispatching the autonomous vehicle to respond to the transportation request based on the assigning.

A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:

• receiving transportation information associated with a transportation request, the transportation information comprising a pick up location and a drop off location;
• determining a first route associated with the transportation request and a non-autonomous vehicle;
• determining a second route associated with the transportation request and an autonomous vehicle based on an operating design domain (ODD) associated with one or more autonomous vehicles in a fleet of vehicles;
• categorizing disengagements in scenarios associated with the ODD into at least an adverse outcome category and a no adverse outcome category based on expected outcomes determined by simulation of the disengagements;
• determining at least one performance metric associated with the second route based on a weighted average of performance metric values associated with road segments of the second route, wherein the performance metric values are based on the expected outcomes of the disengagements in the adverse outcome category, wherein at least one of the road segments has a default performance metric value determined based on other road segments within a threshold distance of the at least one of the road segments, and wherein the performance metric values are weighted based on frequency by which the road segments were traversed;
• selecting the second route based at least in part on the at least one performance metric and a comparison of the first route and the second route;
• assigning an autonomous vehicle from the fleet of vehicles to the transportation request based on selection of the second route;
• and
• dispatching the autonomous vehicle to respond to the transportation request based on the assigning.