Correcting a machine learning model
AssigneeMicrosoft Technology Licensing, LLC
InventorsRyutaro Tanno, Aditya Nori, Melanie Fernandez Pradier, Yingzhen Li
Life of this patent
Estimating the term…
Abstract
A computer-implemented method includes receiving an incorrect prediction output by a trained machine learning model, which has been trained using training data items. The method includes identifying a training data item used to train the model that is a cause of the incorrect prediction, by determining an impact on performance of the trained machine learning model associated with removing the training data item from the plurality of training data. The trained model can then be updated to remove the effect of the identified training data item, allowing the model to be automatically corrected in view of poor quality training data.
Claims20
A system for saving computational resource and memory usage while ensuring that a previous property of a trained machine learning model is retained, the system comprising a processor and a memory, the memory storing computer-readable instructions that, when executed by the processor, cause the system to:
- receive an incorrect prediction of the trained machine learning model, the trained machine learning model based on a plurality of training data items;
- select a training data item of the plurality of training data items;
- calculate an estimate of an impact on performance of the trained machine learning model in respect of the incorrect prediction, the impact caused by removing the selected training data item from the plurality of training data items, wherein the estimate of the impact is calculated by:
- calculating a first posterior probability distribution corresponding to the trained machine learning model;
- calculating a second posterior distribution corresponding to the trained machine learning model with the selected training data item removed;
- and
- comparing the first posterior probability distribution and second posterior probability distribution to determine the estimate of the impact;
- determine that the impact on performance indicates an improvement in performance of the trained machine learning model;
- identify the selected training data item as a cause of the incorrect prediction based on the calculated estimate of the impact on performance indicating improvement in performance of the trained machine learning model;
- and
- automatically correct the trained machine learning model to remove an effect of the selected training data item without requiring entire retraining of the trained machine learning model.
A computer-implemented method for saving computational resource and memory usage while ensuring that a previous property of a trained machine learning model is retained, the method comprising:
- receiving an incorrect prediction of the trained machine learning model, the trained machine learning model based on a plurality of training data items;
- selecting a training data item of the plurality of training data items;
- calculating an estimate of an impact on performance of the trained machine learning model in respect of the incorrect prediction, the impact caused by removing the selected training data item from the plurality of training data items, wherein the estimate of the impact is calculated by:
- calculating a first posterior probability distribution corresponding to the trained machine learning model;
- calculating a second posterior distribution corresponding to the trained machine learning model with the selected training data item removed;
- and
- comparing the first posterior probability distribution and second posterior probability distribution to determine the estimate of the impact;
- determining that the impact on performance indicates an improvement in performance of the trained machine learning model;
- identifying the selected training data item as a cause of the incorrect prediction based on the calculated estimate of the impact on performance indicating improvement in performance of the trained machine learning model;
- and
- automatically correcting the trained machine learning model to remove an effect of the selected training data item without requiring entire retraining of the trained machine learning model.
A tangible non-transient computer-readable storage medium for saving computational resource and memory usage while ensuring that a previous property of a trained machine learning model is retained, the computer-readable storage medium having recorded thereon instructions that, when executed by a computer device cause the computer device to:
- receive an incorrect prediction of the trained machine learning model, the trained machine learning model based on a plurality of training data items;
- select a training data item of the plurality of training data items;
- calculate an estimate of an impact on performance of the trained machine learning model in respect of the incorrect prediction, the impact caused by removing the selected training data item from the plurality of training data items, wherein the estimate of the impact is calculated by:
- calculating a first posterior probability distribution corresponding to the trained machine learning model;
- calculating a second posterior distribution corresponding to the trained machine learning model with the selected training data item removed;
- and
- comparing the first posterior probability distribution and second posterior probability distribution to determine the estimate of the impact;
- determine that the impact on performance indicates an improvement in performance of the trained machine learning model;
- identify the selected training data item as a cause of the incorrect prediction based on the calculated estimate of the impact on performance indicating improvement in performance of the trained machine learning model;
- and
- automatically correct the trained machine learning model to remove an effect of the selected training data item without requiring entire retraining of the trained machine learning model.
Dependencies are inferred from the claim text.