System and method for evaluating the perception system of an autonomous vehicle

A method for optimizing object detection performed by autonomous vehicles, the method comprising:

• receiving, by one or more computing devices having one or more processors, sensor data captured by a sensor of an autonomous vehicle, wherein the sensor data includes one or more first object labels for an object that are applied by one or more systems of the autonomous vehicle;
• receiving, by the one or more computing devices, one or more second object labels for the object that are applied at least in part by a human reviewer using an input device in conjunction with a reviewer object identification application to generate the one or more second object labels;
• comparing, by the one or more computing devices, the one or more first object labels to the one or more second object labels to identify one or more differences between parameters of the one or more first object labels and parameters of the one or more second object labels;
• and
• generating, by the one or more computing devices, a recommendation for an adjustment of at least one object detection parameter of the one or more systems based on the comparing in order to optimize object detection.