Evaluating surrogate machine learning models
AssigneeInternational Business Machines Corporation
InventorsLukasz G. Cmielowski, Wojciech Sobala, Rafal Bigaj, Maksymilian Erazmus
Life of this patent
Estimating the term…
Abstract
A first machine learning model processes a set of inputs to generate a first set of results. Based on that first set of results, a quality control range is calculated. A second machine learning model calculates a mean accuracy of a second set of results, based on the set of inputs. A determination of whether the mean accuracy of the second set of results is within the quality control range is made, and a user is notified of that determination.
Claims20
A method for evaluating machine learning model quality, the method comprising:
- processing, by a first machine learning model, a set of inputs to generate a first set of results;
- calculating, based on the first set of results, a quality control range;
- calculating, by a second machine learning model, a mean accuracy of a second set of results, based on the set of inputs;
- determining whether the mean accuracy of the second set of results is within the quality control range;
- and
- notifying a user of the determination.
A computer program product for evaluating machine learning model quality, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a device to cause the device to:
- process, by a first machine learning model, a set of inputs to generate a first set of results;
- calculate, based on the first set of results, a quality control range;
- calculate, by a second machine learning model, a mean accuracy of a second set of results, based on the set of inputs;
- determine whether the mean accuracy of the second set of results is within the quality control range;
- and
- notify a user of the determination.
A system for evaluating machine learning model quality, comprising:
- a memory with program instructions included thereon;
- and
- a processor in communication with the memory, wherein the program instructions cause the processor to:
- process, by a first machine learning model, a set of inputs to generate a first set of results;
- calculate, based on the first set of results, a quality control range;
- calculate, by a second machine learning model, a mean accuracy of a second set of results, based on the set of inputs;
- determine whether the mean accuracy of the second set of results is within the quality control range;
- and
- notify a user of the determination.
Dependencies are inferred from the claim text.