Back to results
US 2021/0334695 A1Published application
System to correct model drift in machine learning application
AssigneeBank of America Corporation
InventorsUtkarsh Raj, Maharaj Mukherjee
Life of this patent
Estimating the term…
Abstract
A model correction tool automatically detects and corrects model drift in a model for a machine learning application. To detect drift, the tool continuously monitors input data, outputs, and/or technical resources (e.g., processor, memory, network, and input/output resources) used to generate outputs. The tool analyzes changes to input data, outputs, and/or resource usage to determine when drift has occurred. When drift is determined to be occurring, the tool retrains a model for a machine learning application.
Claims20
An apparatus comprising:
- a memory;
- and
- a hardware processor communicatively coupled to the memory, the hardware processor configured to:
- receive an input data into a machine learning application;
- compare the input data with previous input data into the machine learning application to determine a change in a statistical property of the input data;
- receive an output prediction of the machine learning application made by applying a model on the input data;
- determine an accuracy of the model by comparing the output with previous outputs generated by the model;
- determine, based on the change in the statistical property of the input data and the accuracy of the model, a drift score for the model;
- and
- retrain the model when the drift score exceeds a threshold.
7 more elements
A method comprising:
- receiving, by a hardware processor communicatively coupled to a memory, an input data into a machine learning application;
- comparing, by the hardware processor, the input data with previous input data into the machine learning application to determine a change in a statistical property of the input data;
- receiving, by the hardware processor, an output prediction of the machine learning application made by applying a model on the input data;
- determining, by the hardware processor, an accuracy of the model by comparing the output with previous outputs generated by the model;
- determining, by the hardware processor and based on the change in the statistical property of the input data, the accuracy of the model, and the change in processor resources and memory resources used when applying the model, a drift score for the model;
- and
- retraining, by the hardware processor, the model when the drift score exceeds a threshold.
4 more elements
A system comprising:
- a device;
- and
- a model correction tool comprising:
- a memory;
- and
- a hardware processor communicatively coupled to the memory, the hardware processor configured to:
- receive, from the device, an input data into a machine learning application;
- compare the input data with previous input data into the machine learning application to determine a change in a statistical property of the input data;
- receive an output prediction of the machine learning application made by applying a model on the input data;
- determine an accuracy of the model by comparing the output with previous outputs generated by the model;
- determine, based on the change in the statistical property of the input data, the accuracy of the model, and the change in processor resources and memory resources used when applying the model, a drift score for the model;
- and
- retrain the model when the drift score exceeds a threshold.
10 more elements
Dependencies are inferred from the claim text.