Training diverse and robust ensembles of artificial intelligence computer models
AssigneeInternational Business Machines Corporation
InventorsIan Michael Molloy, Taesung Lee, Benjamin James Edwards
Life of this patent
Estimating the term…
Abstract
Mechanisms are provided to implement a hardened ensemble artificial intelligence (AI) model generator. The hardened ensemble AI model generator co-trains at least two AI models. The hardened ensemble AI model generator modifies, based on a comparison of the at least two AI models, a loss surface of one or more of the at least two AI models to prevent an adversarial attack on one AI model, in the at least two AI models, transferring to another AI model in the at least two AI models, to thereby generate one or more modified AI models. At least one of the one or more modified AI models then processes an input to generate an output result.
Claims20
A method, in a data processing system comprising a processor and a memory, the memory comprising instructions which are executed by the processor to specifically configure the processor to implement a hardened ensemble artificial intelligence (AI) model generator, the method comprising:
- co-training, by the hardened ensemble AI model generator, at least two AI models, of an ensemble AI model computing system, on a same domain of training data;
- determining, during co-training of the at least two AI models, a similarity measure between a first gradient of a first AI model in the at least two AI models, and a second gradient of a second AI model in the at least two AI models;
- in response to the similarity measure between the first gradient and the second gradient being equal to or greater than a predetermined threshold similarity measure, determining that a loss surface of one of the first AI model or the second AI model is to be modified;
- in response to determining that the similarity measure is equal to or greater than the predetermined threshold, modifying, by the hardened ensemble AI model generator a loss surface of one of the first AI model or the second AI model, in the at least two AI models, to control a distance between the loss surfaces of the first AI model and the second AI model and prevent an adversarial attack on one AI model transferring to another AI model in the at least two AI models, and thereby generate a modified ensemble of AI models computing system;
- and
- processing, by the modified ensemble of AI models computing system, an input to generate an output result, wherein at least one of the AI models in the modified ensemble of AI models computing system generates a correct output for the output result when the input is an adversarial attack on the AI models.
A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a data processing system, causes the data processing system to implement a hardened ensemble artificial intelligence (AI) model generator that operates to:
- co-train at least two AI models, of an ensemble AI model computing system, on a same domain of training data;
- determine, during co-training of the at least two AI models, a similarity measure between a first gradient of a first AI model in the at least two AI models, and a second gradient of a second AI model in the at least two AI models;
- determine, in response to the similarity measure between the first gradient and the second gradient being equal to or greater than a predetermined threshold similarity measure, that a loss surface of one of the first AI model or the second AI model is to be modified;
- modify, in response to determining that the similarity measure is equal to or greater than the predetermined threshold, a loss surface of one of the first AI model or the second AI model, in the at least two AI models, to control a distance between the loss surfaces of the first AI model and the second AI model and prevent an adversarial attack on one AI model transferring to another AI model in the at least two AI models, and thereby generate a modified ensemble of AI models computing system;
- and
- process, by the modified ensemble of AI models computing system, an input to generate an output result, wherein at least one of the AI models in the modified ensemble of AI models computing system generates a correct output for the output result when the input is an adversarial attack on the AI models.
An apparatus comprising:
- at least one processor;
- and
- at least one memory coupled to the at least one processor, wherein the at least one memory comprises instructions which, when executed by the at least one processor, cause the at least one processor to implement a hardened ensemble artificial intelligence (AI) model generator that operates to:
- co-train at least two AI models, of an ensemble AI model computing system, on a same domain of training data;
- determine, during co-training of the at least two AI models, a similarity measure between a first gradient of a first AI model in the at least two AI models, and a second gradient of a second AI model in the at least two AI models;
- determine, in response to the similarity measure between the first gradient and the second gradient being equal to or greater than a predetermined threshold similarity measure, that a loss surface of one of the first AI model or the second AI model is to be modified;
- modify, in response to determining that the similarity measure is equal to or greater than the predetermined threshold, a loss surface of one of the first AI model or the second AI model, in the at least two AI models, to control a distance between the loss surfaces of the first AI model and the second AI model and prevent an adversarial attack on one AI model transferring to another AI model in the at least two AI models, and thereby generate a modified ensemble of AI models computing system;
- and
- process, by the modified ensemble of AI models computing system, an input to generate an output result, wherein at least one of the AI models in the modified ensemble of AI models computing system generates a correct output for the output result when the input is an adversarial attack on the AI models.
Dependencies are inferred from the claim text.