# US 2026/0064849 A1 · Real-Time Artificial Intelligence Model Vulnerability Testing System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0064849 A1
- Document ID: US20260064849A1
- Status: Published application
- Kind: A1
- Application number: 18/823,240
- Filed: 2024-09-03
- Published: 2026-03-05
- Earliest claimed: 2024-09-03
- Assignees: Bank of America Corporation
- Applicants: Bank of America Corporation
- Inventors: Shailendra Singh; Sindu Puthenvarium Sreedevi; Vasudev Bhanu Rasam
- Web page: https://patentgrep.com/patent/US20260064849A1

## CPC

- G06F 21/577: {Assessing vulnerabilities and evaluating computer system security}
- G06F 2221/033: Test or assess software
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Various aspects of the disclosure relate to automated real-time penetration testing of artificial intelligence (AI) models used by AI-based applications. A real-time AI model penetration testing plugin orchestrates real-time penetration testing for scanning and detecting vulnerabilities in AI model-based applications, particularly those applications leveraging generative AI algorithms. A plugin installation on a user device automatically scans AI model-based application operations hosted locally to the user device and/or centrally located on a remote server and provides a degree of confidence and trust corresponding to use of the AI model-based application. The plugin orchestrates security scans before all transactions and/or tasks executed by the AI application and initiates a security response based on a trust score corresponding to penetration test results.

## Claims

### Claim 1

A system comprising:
a host computing device hosting an artificial intelligence (AI) model management system, wherein an AI model managed by the AI model management system;
a computing device, comprising:
a processor; and
memory storing computer-readable instructions that, when executed by the processor, cause the computing device to:
enable a real-time AI model scanning plugin monitoring operations of an AI-based application, wherein the AI-based application comprises the AI model;
capture, by the real-time AI model scanning plugin, a first command to the AI-based application;
trigger, by the real-time AI model scanning plugin, a real-time penetration test of the AI model by the host computing device, wherein the host computing device calculates a trust score based on results of the real-time penetration test; and
disable, based on a trust score meeting a first threshold, operation of the AI-based application.

### Claim 2 (depends on 1)

The system of claim 1, wherein the host computing device comprises a machine learning operations (MLOps) system.

### Claim 3 (depends on 1)

The system of claim 1, wherein the real-time AI model scanning plugin comprises a plugin to the AI-based application.

### Claim 4 (depends on 1)

The system of claim 1, wherein the real-time AI model scanning plugin comprises a framework within which the AI-based application operates.

### Claim 5 (depends on 1)

The system of claim 1, wherein the first command comprises a command to enable operation of the AI model.

### Claim 6 (depends on 1)

The system of claim 1, wherein the instructions cause the computing device to partially disable, based on the trust score meeting a second threshold condition, a set of operations of the AI-based application wherein the first command requests an operation of the set of operations.

### Claim 7 (depends on 1)

The system of claim 1, wherein the instructions cause the API route testing platform to enable, based on the trust score meeting a third threshold condition, unrestricted operation the AI-based application in response to the first command.

### Claim 8

A method comprising:
enabling, on a user device, a real-time AI model scanning plugin that monitors operations of an AI-based application, wherein the AI-based application comprises an AI model;
capturing, by the real-time AI model scanning plugin, a first command to the AI-based application;
triggering, by the real-time AI model scanning plugin, a real-time penetration test of the AI model by a host computing device, wherein the host computing device calculates a trust score based on results of the real-time penetration test;
disabling, based on the trust score meeting a first threshold, operation of the AI-based application; and
capturing, by the real-time AI model scanning plugin, a next command to the AI-based application and repeating the triggering and disabling steps in real-time.

### Claim 9 (depends on 8)

The method of claim 8, wherein the host computing device comprises a machine learning operations (MLOps) system.

### Claim 10 (depends on 8)

The method of claim 8, wherein the real-time AI model scanning plugin comprises a plugin to the AI-based application.

### Claim 11 (depends on 8)

The method of claim 8, wherein the real-time AI model scanning plugin comprises a framework within which the AI-based application operates.

### Claim 12 (depends on 8)

The method of claim 8, wherein the first command comprises a command to enable operation of the AI model.

### Claim 13 (depends on 12)

The method of claim 12, further comprising partially disabling, based on the trust score meeting a second threshold condition, a set of operations of the AI-based application wherein the first command requests an operation of the set of operations.

### Claim 14 (depends on 13)

The method of claim 13, further comprising enabling, based on the trust score meeting a third threshold condition, unrestricted operation the AI-based application in response to the first command.

### Claim 15

Non-transitory computer readable media storing instructions that, when executed by a processor, cause a computing device to:
enable a real-time AI model scanning plugin monitoring operations of an AI-based application, wherein the AI-based application comprises an AI model;
capture, by the real-time AI model scanning plugin, a first command to the AI-based application;
trigger, by the real-time AI model scanning plugin, a real-time penetration test of the AI model by a host computing device, wherein the host computing device calculates a trust score based on results of the real-time penetration test; and
disable, by the real-time AI model scanning plugin, based on a trust score meeting a first threshold, operation of the AI-based application.

### Claim 16 (depends on 15)

The non-transitory computer readable media of claim 15, wherein the host computing device comprises a machine learning operations (MLOps) system.

### Claim 17 (depends on 16)

The non-transitory computer readable media of claim 16, wherein the real-time AI model scanning plugin comprises a plugin to the AI-based application.

### Claim 18 (depends on 15)

The non-transitory computer readable media of claim 15, wherein the real-time AI model scanning plugin comprises a framework within which the AI-based application operates.

### Claim 19 (depends on 15)

The non-transitory computer readable media of claim 15, wherein the instructions cause the computing device to disable, based on the trust score meeting a second threshold condition, a set of operations of the AI-based application wherein the first command requests an operation of the set of operations.

### Claim 20 (depends on 15)

The non-transitory computer readable media of claim 15, wherein the instructions cause the computing device to enable, based on the trust score meeting a third threshold condition, unrestricted operation the AI-based application in response to the first command.
