Visual analytics platform for updating object detection models in autonomous driving applications

A method for updating an object detection model for autonomous driving with an object detection model visual analysis platform, the method comprising:

• extracting, with an electronic processor and a first object detection model for autonomous driving, object information from image data;
• extracting, with the electronic processor, characteristics of objects from metadata associated with the image data;
• generating, with the electronic processor, a summary of the object information and the characteristics that are extracted;
• generating, with the electronic processor, coordinated visualizations based on the summary and the characteristics that are extracted;
• outputting, with the electronic processor, the coordinated visualizations for display on a display device;
• receiving, with the electronic processor, a first one or more user inputs from an interface device via a communication interface, the first one or more user inputs selecting a portion of information that is included in the coordinated visualizations;
• generating, with the electronic processor, a recommendation graphical user interface element based on the coordinated visualizations and the first one or more user inputs, the recommendation graphical user interface element summarizing one or more potential weaknesses of the first object detection model;
• outputting, with the electronic processor, the recommendation graphical user interface element for display on the display device;
• receiving, with the electronic processor, a second user input selecting one or more individual objects from the object information that is extracted and included in the one or more potential weaknesses;
• outputting, with the electronic processor, an image based on the image data and the second user input for display on the display device, the image highlighting the one or more individual objects;
• receiving, with the electronic processor, a third user input classifying the one or more individual objects as an actual weakness with respect to the first object detection model;
• and
• updating, with the electronic processor, the first object detection model based at least in part on the classification of the one or more individual objects as the actual weakness with respect to the first object detection model to generate a second object detection model for autonomous driving.

A non-transitory computer-readable medium comprising instructions that, when executed by an electronic processor, causes the electronic processor to perform a set of operations comprising:

• extracting object information from image data with a first object detection model for autonomous driving;
• extracting characteristics of objects from metadata associated with the image data;
• generating a summary of the object information and the characteristics that are extracted;
• generating coordinated visualizations based on the summary and the characteristics that are extracted;
• outputting the coordinated visualizations for display on a display device;
• receiving a first one or more user inputs from an interface device via a communication interface, the first one or more user inputs selecting a portion of information that is included in the coordinated visualizations;
• generating a recommendation graphical user interface element based on the coordinated visualizations and the first one or more user inputs, the recommendation graphical user interface element summarizing one or more potential weaknesses of the first object detection model;
• outputting the recommendation graphical user interface element for display on the display device;
• receiving a second user input selecting one or more individual objects from the object information that is extracted and included in the one or more potential weaknesses;
• outputting an image based on the image data and the second user input for display on the display device, the image highlighting the one or more individual objects;
• receiving a third user input classifying the one or more individual objects as an actual weakness with respect to the first object detection model;
• and
• updating the first object detection model based at least in part on the classification of the one or more individual objects as the actual weakness with respect to the first object detection model to generate a second object detection model for autonomous driving.