Controlling performance of deployed deep learning models on resource constrained edge device via predictive models

A computer-implemented method, comprising:

• monitoring, via a processor on a fog layer used to orchestrate a resource-constrained edge device, an input data and a predicted performance of a deep learning (DL) model deployed on the resource constrained edge device from a predictive model trained to predict a performance characteristic of the deployed DL model on the resource constrained edge device, wherein the predictive model is trained based on training data and output from a performance estimator that estimates a performance of the deployed DL based on the training data and output from the DL model generated on the training data, wherein the predictive model is on the resource constrained edge device and wherein the predicted performance is a frames per second at which the DL model runs on the resource constrained edge device;
• redirecting, by the processor, the input data from the deployed DL model on the resource-constrained edge device to the predictive model on the resource-constrained edge device;
• modifying, via the processor, a control input based on the input data and the predicted performance of the deployed DL model;
• and
• sending, via the processor, the modified control input to the deployed DL model.

A computer program product for controlling deployed deep learning (DL) models, the computer program product comprising a computer-readable storage medium having program code embodied therewith, the program code executable by a processor to cause the processor to:

• monitor, at a fog layer used to orchestrate a resource-constrained edge device, an input data and a predicted performance of a DL model deployed on the resource constrained edge device from a predictive model trained to predict a performance characteristic of the deployed DL model on the resource constrained edge device, wherein the predictive model is trained based on training data and output from a performance estimator that estimates a performance of the deployed DL based on the training data and output from the DL model generated on the training data, and wherein the predicted performance comprises a battery drain on the resource constrained edge device, and wherein the predictive model is on the resource constrained edge device and wherein the resource-constrained edge device is not part of the fog layer;
• redirect, by the fog layer, the input data to the predictive model;
• modify a control input based on the input data and the predicted performance of the deployed DL model;
• and
• send, by the fog layer, the modified control input to the deployed DL model.