# US 2025/0148347 A1 · System and method to detect manipulation of training data used for machine learning models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148347 A1
- Document ID: US20250148347A1
- Status: Published application
- Kind: A1
- Application number: 18/387,134
- Filed: 2023-11-06
- Published: 2025-05-08
- Earliest claimed: 2023-11-06
- Assignees: Bank of America Corporation
- Applicants: Bank of America Corporation
- Inventors: Saurabh Arora; Srija Reddy Kallu; Durga Prasad Kutthumolu
- Web page: https://patentgrep.com/patent/US20250148347A1

## CPC

- G06N 20/00: Machine learning
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/08: Learning methods
- G06N 5/04: Inference or reasoning models
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems, computer program products, and methods are described herein for detecting manipulation of training data used for machine learning models. The present disclosure is configured to receive an interaction originating from an end-point device; validate the interaction via a primary model; identify a set of triggers within a backdoor model, where the backdoor model is modeled off the primary model capable of undergoing stress testing associated with a set of triggers; pause the interaction upon identification of the set of triggers within the backdoor model; transmit the identified set of triggers to an autonomous artificial general intelligence (AGI) agent, generate a set of synthetic data via the autonomous AGI agent, where the set of synthetic data removes the set of triggers from the primary model; and distribute the set of synthetic data to the primary model to correct the set of triggers within the primary model.

## Claims

### Claim 1

A system to detect manipulation of training data used for machine learning models, the system comprising:
at least one non-transitory storage device; and
at least one processing device coupled to the at least one non-transitory storage device, wherein the at least one processing device is configured to:
receive an interaction originating from an end-point device;
validate the interaction via a primary model;
identify a set of triggers within a backdoor model;
wherein the backdoor model is modeled off of the primary model capable of undergoing stress testing associated with a set of triggers;
pause the interaction upon identification of the set of triggers within the backdoor model;
transmit the identified set of triggers to an autonomous artificial general intelligence (AGI) agent;
generate a set of synthetic data via the autonomous AGI agent,
wherein the set of synthetic data is training data that removes the set of triggers from the primary model; and
distribute the set of synthetic data to the primary model to correct the set of triggers within the primary model.

### Claim 2 (depends on 1)

The system of claim 1, wherein the at least one processing device is further configured to:
transmit the identified set of triggers via the autonomous AGI agent to an overseer AGI agent; and
distribute the set of triggers via the overseer AGI agent to a plurality of autonomous AGI agents connected to the overseer AGI agent,
wherein the plurality of autonomous AGI agents generates a set of synthetic data for a respective primary model to correct the identified set of triggers.

### Claim 3 (depends on 2)

The system of claim 2, wherein the overseer AGI agent determines a set of autonomous AGI agents within the plurality of autonomous AGI agents connected to the overseer AGI agent which may receive the set of triggers.

### Claim 4 (depends on 1)

The system of claim 1, wherein identification of the set of triggers within the backdoor model are identified using a latent space outlier technique.

### Claim 5 (depends on 1)

The system of claim 1, wherein identification of the set of triggers within the backdoor model are identified using an input space outlier technique.

### Claim 6 (depends on 1)

The system of claim 1, wherein validation of the interaction via the primary model further comprises:
pause the interaction originating from the end-point device; and
transmit a notification to the end-point device.

### Claim 7 (depends on 1)

The system of claim 1, wherein the backdoor model receives a refined set of inputs comprised of potential triggers.

### Claim 8

A computer program product to detect manipulation of training data used for machine learning models, wherein the computer program product comprises at least one non-transitory computer-readable medium having computer-readable program code portions embodied there, the computer-readable program code portions which when executed by a processing device are configured to cause the processor to perform the following operations:
receive an interaction originating from an end-point device;
validate the interaction via a primary model;
identify a set of triggers within a backdoor model,
wherein the backdoor model is modeled off of the primary model capable of undergoing stress testing associated with a set of triggers;
pause the interaction upon identification of the set of triggers within the backdoor model;
transmit the identified set of triggers to an autonomous artificial general intelligence (AGI) agent;
generate a set of synthetic data via the autonomous AGI agent,
wherein the set of synthetic data is training data that removes the set of triggers from the primary model; and
distribute the set of synthetic data to the primary model to correct the set of triggers within the primary model.

### Claim 9 (depends on 8)

The computer program product of claim 8, wherein the processor further performs the following operations:
transmit the identified set of triggers via the autonomous AGI agent to an overseer AGI agent; and
distribute the set of triggers via the overseer AGI agent to a plurality of autonomous AGI agents connected to the overseer AGI agent,
wherein the plurality of autonomous AGI agents generates a set of synthetic data for a respective primary model to correct the identified set of triggers.

### Claim 10 (depends on 9)

The computer program product of claim 9, wherein the overseer AGI agent determines a set of autonomous AGI agents within the plurality of autonomous AGI agents connected to the overseer AGI agent which may receive the set of triggers.

### Claim 11 (depends on 8)

The computer program product of claim 8, wherein identification of the set of triggers within the backdoor model are identified using a latent space outlier technique.

### Claim 12 (depends on 8)

The computer program product of claim 8, wherein identification of the set of triggers within the backdoor model are identified using an input space outlier technique.

### Claim 13 (depends on 8)

The computer program product of claim 8, wherein validation of the interaction via the primary model further comprises:
pause the interaction originating from the end-point device; and
transmit a notification to the end-point device.

### Claim 14 (depends on 8)

The computer program product of claim 8, wherein the backdoor model receives a refined set of inputs comprised of potential triggers.

### Claim 15

A computer-implemented method for detecting manipulation of training data used for machine learning models, the computer-implemented method comprising:
receiving an interaction originating from an end-point device;
validate the interaction via a primary model;
identifying a set of triggers within a backdoor model,
wherein the backdoor model is modeled off of the primary model capable of undergoing stress testing associated with a set of triggers;
pausing the interaction upon identification of the set of triggers within the backdoor model;
transmitting the identified set of triggers to an autonomous artificial general intelligence (AGI) agent;
generating a set of synthetic data via the autonomous AGI agent,
wherein the set of synthetic data is training data that removes the set of triggers from the primary model; and
distributing the set of synthetic data to the primary model to correct the set of triggers within the primary model.

### Claim 16 (depends on 15)

The computer-implemented method of claim 15, wherein the computer-implemented method further comprises:
transmitting the identified set of triggers via the autonomous AGI agent to an overseer AGI agent; and
distribute the set of triggers via the overseer AGI agent to a plurality of autonomous AGI agents connected to the overseer AGI agent,
wherein the plurality of autonomous AGI agents generates a set of synthetic data for a respective primary model to correct the identified set of triggers.

### Claim 17 (depends on 16)

The computer-implemented method of claim 16, wherein the overseer AGI agent determines a set of autonomous AGI agents within the plurality of autonomous AGI agents connected to the overseer AGI agent which may receive the set of triggers.

### Claim 18 (depends on 15)

The computer-implemented method of claim 15, wherein identifying the set of triggers within the backdoor model are identified using a latent space outlier technique.

### Claim 19 (depends on 15)

The computer-implemented method of claim 15, wherein identifying the set of triggers within the backdoor model are identified using an input space outlier technique.

### Claim 20 (depends on 15)

The computer-implemented method of claim 15, wherein validating the interaction via the primary model further comprises:
pausing the interaction originating from the end-point device; and
transmitting a notification to the end-point device.
