Systems and methods for reviewing performance of computer models for safety analysis in transportation services

A method for reviewing performance of computer models for safety analysis in transportation services, comprising:

• receiving, by a communication interface, log data associated with at least one reported safety event, wherein the log data includes information of the reported safety event, a transportation service during which the reported safety event occurs, and one or more computer models used for predicting the reported safety event, wherein the one or more computer models are associated with a first feature pattern and a first model performance;
• extracting, by at least one processor, a plurality of features from the log data that are used by the computer models to make a safety prediction;
• determining, by the at least one processor, a second feature pattern indicative of respective impacts of the features on the safety prediction result made by the computer models;
• determining, by the at least one processor, a second model performance of the computer models based on the safety prediction made by the computer models compared with the reported safety event;
• detecting a change in feature pattern based on the first feature pattern and the second feature pattern or a performance degradation based on the first model performance and the second model performance;
• and
• generating, by the at least one processor, an alert to upgrade the computer models.

A non-transitory computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by at least one processor, performs a method for reviewing performance of computer models for safety analysis in transportation services, the method comprising:

• receiving log data associated with at least one reported safety event, wherein the log data includes information of the reported safety event, a transportation service during which the reported safety event occurs, and one or more computer models used for predicting the reported safety event, wherein the one or more computer models are associated with a first feature pattern and a first model performance;
• extracting a plurality of features from the log data that are used by the computer models to make a safety prediction;
• determining a second feature pattern indicative of respective impacts of the features on the safety prediction result made by the computer models;
• determining a second model performance of the computer models based on the safety prediction made by the computer models compared with the reported safety event;
• detecting a change in feature pattern based on the first feature pattern and the second feature pattern or a performance degradation based on the first model performance and the second model performance;
• and
• generating an alert to upgrade the computer models.