# US 12,223,419 B2 · Controlling performance of deployed deep learning models on resource constrained edge device via predictive models > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,419 B2 - Document ID: US12223419B2 - Status: Granted - Kind: B2 - Application number: 16/550,290 - Filed: 2019-08-26 - Published: 2021-03-04 - Granted: 2025-02-11 - Earliest claimed: 2019-08-26 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Lior Turgeman - Web page: https://patentgrep.com/patent/US12223419B2 ## CPC - G06N 3/08: Learning methods - G06F 8/60: Software deployment - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06N 3/10: Interfaces, programming languages or software development kits, e.g. for simulating neural networks - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An example system includes a processor to receive a data input and a predicted performance of a Deep Learning (DL) model deployed on a resource constrained edge device from a predictive model. The processor is to modify a control input for the DL model based on the data input and the predicted performance. The processor is to send the control input to the deployed DL model to modify performance of the DL model. ## Claims ### Claim 1 A system, comprising a hardware processor to: receive, at a fog layer used to orchestrate a resource-constrained edge device, a data input and a predicted performance of a deep learning (DL) model deployed on the resource constrained edge device from a predictive model trained to predict a performance characteristic of the deployed DL model on the resource constrained edge device, wherein the predictive model is trained based on training data and output from a performance estimator that estimates a performance of the deployed DL based on the training data and output from the DL model generated on the training data, wherein the predictive model is on the resource constrained edge device and wherein the resource-constrained edge device is not part of the fog layer; redirect, by the fog layer, the data input to the predictive model; modify a control input for the deployed DL model based on the data input and the predicted performance; and send, by the fog layer, the modified control input to the deployed DL model to modify performance of the deployed DL model on the resource constrained edge device. ### Claim 2 (depends on 1) The system of claim 1, wherein the hardware processor is to train the predictive model based on a predictive error between a performance estimation and output of the predictive model, wherein the output of the predictive model is based on a training input, and the performance estimation is based on the training input and an output of the deployed DL model. ### Claim 3 (depends on 1) The system of claim 1, wherein the predicted performance comprises a runtime performance, battery drain on the resource constrained edge device, a memory usage, an accuracy, or any combination thereof. ### Claim 4 (depends on 1) The system of claim 1, wherein the hardware processor is to modify the control input based on the predicted performance in response to detecting a change to the data input that exceeds a threshold value. ### Claim 5 (depends on 1) The system of claim 1, wherein the hardware processor is to modify the control input based on the predicted performance in response to detecting that the predicted performance comprises a predicted failure of the deployed DL model. ### Claim 6 (depends on 1) The system of claim 1, wherein the control input is modified to optimize a predefined performance criterion over a specified time horizon. ### Claim 7 (depends on 1) The system of claim 1, wherein the hardware processor is to switch the receiving of the control input between a data buffer, the deployed model, and the predictive model. ### Claim 8 A computer-implemented method, comprising: monitoring, via a processor on a fog layer used to orchestrate a resource-constrained edge device, an input data and a predicted performance of a deep learning (DL) model deployed on the resource constrained edge device from a predictive model trained to predict a performance characteristic of the deployed DL model on the resource constrained edge device, wherein the predictive model is trained based on training data and output from a performance estimator that estimates a performance of the deployed DL based on the training data and output from the DL model generated on the training data, wherein the predictive model is on the resource constrained edge device and wherein the predicted performance is a frames per second at which the DL model runs on the resource constrained edge device; redirecting, by the processor, the input data from the deployed DL model on the resource-constrained edge device to the predictive model on the resource-constrained edge device; modifying, via the processor, a control input based on the input data and the predicted performance of the deployed DL model; and sending, via the processor, the modified control input to the deployed DL model. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, wherein the control input is modified in response to detecting a change in the input data that exceeds a threshold change. ### Claim 10 (depends on 8) The computer-implemented method of claim 8, wherein the control input is modified in response to detecting a change in the predicted performance of the deployed DL model, wherein the change in the predicted performance is an increase in the frames per second at which the DL model runs. ### Claim 11 (depends on 8) The computer-implemented method of claim 8, wherein modifying the control input comprises modifying a parameter of the deployed DL model to optimize a predefined performance criterion over a specified time horizon. ### Claim 12 (depends on 8) The computer-implemented method of claim 8, comprising generating an alert in response to detecting a change in the predicted performance of the deployed DL model or a predicted failure of the deployed DL model. ### Claim 13 (depends on 8) The computer-implemented method of claim 8, comprising training the predictive model based on a predictive error between a performance estimation and the predicted performance of the predictive model, wherein the performance estimation is based on a training input and an output performance characteristic of the deployed DL model, and the predicted performance of the predictive model is based on the training input and the performance estimation. ### Claim 14 (depends on 13) The computer-implemented method of claim 13, wherein the predictive model is modified based on the predictive error until the predictive error does not exceed a threshold error. ### Claim 15 A computer program product for controlling deployed deep learning (DL) models, the computer program product comprising a computer-readable storage medium having program code embodied therewith, the program code executable by a processor to cause the processor to: monitor, at a fog layer used to orchestrate a resource-constrained edge device, an input data and a predicted performance of a DL model deployed on the resource constrained edge device from a predictive model trained to predict a performance characteristic of the deployed DL model on the resource constrained edge device, wherein the predictive model is trained based on training data and output from a performance estimator that estimates a performance of the deployed DL based on the training data and output from the DL model generated on the training data, and wherein the predicted performance comprises a battery drain on the resource constrained edge device, and wherein the predictive model is on the resource constrained edge device and wherein the resource-constrained edge device is not part of the fog layer; redirect, by the fog layer, the input data to the predictive model; modify a control input based on the input data and the predicted performance of the deployed DL model; and send, by the fog layer, the modified control input to the deployed DL model. ### Claim 16 (depends on 15) The computer program product of claim 15, further comprising program code executable by the processor to modify the control input based on the predicted performance in response to detecting a change to the input data that exceeds a threshold value. ### Claim 17 (depends on 15) The computer program product of claim 15, further comprising program code executable by the processor to modify the control input based on the predicted performance in response to detecting that the predicted performance comprises a predicted failure of the deployed DL model. ### Claim 18 (depends on 15) The computer program product of claim 15, further comprising program code executable by the processor to modify the control input to optimize a predefined performance criterion over a specified time horizon. ### Claim 19 (depends on 15) The computer program product of claim 15, further comprising program code executable by the processor to generate an alert in response to detecting a change in the predicted performance of the deployed DL model or a predicted failure of the deployed DL model. ### Claim 20 (depends on 15) The computer program product of claim 15, further comprising program code executable by the processor to train the predictive model based on predictive error between a performance estimation and output of the predictive model, wherein the output of the predictive model is based on a training input, and the performance estimation is based on the training input and an output of the deployed DL model.