Framework for testing machine learning workflows

A system, comprising:

• one or more processors;
• and
• memory storing instructions that, when executed by the one or more processors, cause the system to:
• obtain a configuration for a staging test of a machine learning model, wherein the configuration comprises a model name for the machine learning model, a duration of the staging test, and a use case associated with the machine learning model;
• select a staging test host for the staging test;
• deploy the staging test on the staging test host in a staging environment, wherein the deployed staging test executes the machine learning model based on live traffic received from a production environment;
• and
• after the staging test has completed, output a set of metrics representing a system impact of the machine learning model on the staging test host.

A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:

• obtaining a configuration for a staging test of a machine learning model, wherein the configuration comprises a model name for the machine learning model, a duration of the staging test, and a use case associated with the machine learning model;
• selecting a staging test host for the staging test;
• deploying the staging test on the staging test host in a staging environment, wherein the deployed staging test executes the machine learning model based on live traffic received from a production environment;
• and
• after the staging test has completed, outputting a set of metrics representing a system impact of the machine learning model on the staging test host.