Automated regression detection framework for supporting robust version changes of machine learning applications

A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for regression detection in machine learning (ML) models resulting from version changes in an application leveraging the ML models, the operations comprising:

• receiving a project structure representing a regression test file directory for regression inference and comprising a set of test scenarios;
• determining that a test scenario of the set of test scenarios is to be executed;
• transmitting a request for a test inference job to be executed using a second version of the application, the test inference job representing the test scenario;
• receiving a set of actual results of the test inference job;
• calculating a prediction score based on the set of actual results and a set of expected results of the test scenario;
• and
• selectively indicating regression of the one or more ML models of the test scenario based on the prediction score.

A system, comprising:

• a computing device;
• and
• a computer-readable storage device coupled to the computing device and having instructions stored thereon which, when executed by the computing device, cause the computing device to perform operations for regression detection in machine learning (ML) models resulting from version changes in an application leveraging the ML models, the operations comprising:
• receiving a project structure representing a regression test file directory for regression inference and comprising a set of test scenarios;
• determining that a test scenario of the set of test scenarios is to be executed;
• transmitting a request for a test inference job to be executed using a second version of the application, the test inference job representing the test scenario;
• receiving a set of actual results of the test inference job;
• calculating a prediction score based on the set of actual results and a set of expected results of the test scenario;
• and
• selectively indicating regression of the one or more ML models of the test scenario based on the prediction score.