Recommended configurations of machine learning computing resources

A method, comprising:

• monitoring a machine learning model deployed on one or more computing resources that use the machine learning model to generate respective inferences according to different inference requests, wherein the monitoring comprises:
• recording utilization metrics of the one or more computing resources performing the different inference requests;
• recording the different inference requests;
• and
• recording the respective inferences;
• generating a recommended computing resource configuration for the machine learning model based, at least in part, on the utilization metrics of the one or more computing resources and an optimization objective for utilizing computing resources;
• making a determination to provide the recommended computing resource configuration based, at least in part, on an accuracy analysis performed for the machine learning model deployed on the recommended computing resource configuration using the recorded different inference requests and the recorded respective inferences;
• and
• providing the recommended computing resource configuration for the deployed machine learning model.

One or more computer-readable storage media storing instructions that, when executed on or across one or more processors, cause the one or more processors to:

• monitor a machine learning model deployed on one or more computing resources that use the machine learning model to generate respective inferences according to different inference requests, wherein the monitoring comprises:
• recording utilization metrics of the one or more computing resources performing the different inference requests;
• recording the different inference requests;
• and
• recording the respective inferences;
• generate a recommended computing resource configuration for the machine learning model based, at least in part, on the utilization metrics of the one or more computing resources and an optimization objective for utilizing computing resources;
• determine to provide the recommended computing resource configuration based, at least in part, on an accuracy analysis performed for the machine learning model deployed on the recommended computing resource configuration using the recorded different inference requests and the recorded respective inferences;
• and
• provide the recommended computing resource configuration for the deployed machine learning model.