US 2025/0080587 A1 · MANAGING ARTIFICIAL INTELLIGENCE MODELS TO IDENTIFY GOALS OF MALICIOUS ATTACKERS

Abstract

Methods and systems for managing an artificial intelligence (AI) model are disclosed. An AI model may be part of an evolving AI model pipeline, the processes of which may include obtaining training data from data sources used to update the AI model. An attacker may introduce poisoned training data via one or more of the data sources as a form of attack on the AI model. When the poisoned training data is identified, the poisoned training data may be compared to existing training data to determine the attacker's goal. Based on the attacker's goal, remedial actions may be performed that may update operation of pipeline. The updated operation of the pipeline may reduce the computational expense for remediating impact of the poisoned training data, and may reduce the likelihood of obtaining poisoned training data in the future.

Claims20

Dependent claims

A method for managing an artificial intelligence (AI) model, comprising:

making an identification that a portion of new training data for the AI model is poisoned training data, the AI model being part of an evolving AI model pipeline;
and
based on the identification:
identifying a portion of existing training data on which the portion of the new training data is based;
for a sample of the portion of the new training data, identifying at least one sample of the existing training data upon which the sample of the portion of the new training data is based;
identifying at least one label for the at least one sample;
identifying a goal of a malicious entity using the at least one label for the at least one sample;
and
performing a remedial action set to update operation of the evolving AI model pipeline based on the goal.

6 more elements

A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for managing an artificial intelligence (AI) model, the operations comprising:

making an identification that a portion of new training data for the AI model is poisoned training data, the AI model being part of an evolving AI model pipeline;
and
based on the identification:
identifying a portion of existing training data on which the portion of the new training data is based;
for a sample of the portion of the new training data, identifying at least one sample of the existing training data upon which the sample of the portion of the new training data is based;
identifying at least one label for the at least one sample;
identifying a goal of a malicious entity using the at least one label for the at least one sample;
and
performing a remedial action set to update operation of the evolving AI model pipeline based on the goal.

6 more elements

A data processing system, comprising:

a processor;
and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for managing an artificial intelligence (AI) model, the operations comprising:
making an identification that a portion of new training data for the AI model is poisoned training data, the AI model being part of an evolving AI model pipeline;
and
based on the identification:
identifying a portion of existing training data on which the portion of the new training data is based;
for a sample of the portion of the new training data, identifying at least one sample of the existing training data upon which the sample of the portion of the new training data is based;
identifying at least one label for the at least one sample;
identifying a goal of a malicious entity using the at least one label for the at least one sample;
and
performing a remedial action set to update operation of the evolving AI model pipeline based on the goal.

9 more elements

Dependencies are inferred from the claim text.

Managing artificial intelligence models to identify goals of malicious attackers

Life of this patent

Managing artificial intelligence models to identify goals of malicious attackers

Life of this patent

Abstract

Claims20