# US 2026/0099593 A1 · METHODS, SYSTEMS, AND COMPUTER READABLE MEDIA FOR DEFENDING LARGE LANGUAGE MODELS (LLMs) AGAINST JAILBREAKING ATTACKS

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0099593 A1
- Document ID: US20260099593A1
- Status: Published application
- Kind: A1
- Application number: 18/907,376
- Filed: 2024-10-04
- Published: 2026-04-09
- Earliest claimed: 2024-10-04
- Assignees: The Trustees of the University of Pennsylvania
- Applicants: The Trustees of the University of Pennsylvania
- Inventors: George J. Pappas; Alexander Beck Robey; Seyed Hamed Hassani; Eric Wong
- Web page: https://patentgrep.com/patent/US20260099593A1

## CPC

- G06F 21/554: {involving event detection and direct action}
- G06F 2221/033: Test or assess software
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method for defending an LLM against a jailbreaking attack includes receiving an input prompt for generating output from the LLM. The method further includes generating N versions of the input prompt, wherein generating each of the N versions includes perturbing the input prompt to generate N perturbed input prompts. The method further includes providing the N perturbed input prompts as input to the LLM, which generates N responses. The method further includes aggregating the responses by estimating a group effect of the perturbed input prompts on the LLM. The method further includes selecting, as output, one of the N responses corresponding to a perturbed input prompt that causes the LLM to generate a response that agrees with the estimated group effect.

## Claims

### Claim 1

A method for defending a large language model (LLM) against a jailbreaking attack, the method comprising:
receiving an input prompt for generating output from the LLM, wherein the input prompt is a jailbreaking attack and includes a goal string requesting objectionable content from the LLM and an adversarial portion that when added to the goal string is intended to cause the LLM to output the objectionable content and thereby jailbreak the LLM;
generating N versions, N being an integer of at least two, of the input prompt, wherein generating, without knowledge of a position or presence of the adversarial portion of the input prompt, each of the N versions includes perturbing, using a randomizing function, the input prompt to generate N perturbed input prompts;
providing the N perturbed input prompts as input to the LLM, which generates N responses;
aggregating the responses by estimating a majority effect of the perturbed input prompts on the LLM, wherein estimating the majority effect includes evaluating, for each of the responses, a function determines whether or not the response includes the objectionable content and determining, using the function, that a majority of the responses do not include the objectionable content; and
selecting, as output, one of the responses in the majority that does not include the objectionable content, thereby nullifying the jailbreaking attack.

### Claim 2 (depends on 1)

The method of claim 1 wherein perturbing the input prompts includes adding characters to each of the input prompts.

### Claim 3 (depends on 2)

The method of claim 2 wherein adding the characters to each of the input prompts includes using the randomizing function to select characters in each of the input prompts and adding a character sampled from an alphabet of characters after each of the selected characters.

### Claim 4 (depends on 1)

The method of claim 1 perturbing the input prompts includes replacing characters in each of the input prompts.

### Claim 5 (depends on 4)

The method of claim 4 wherein replacing the characters includes using the randomizing function to select character locations in each of the input prompts and replacing characters at the selected locations with characters sampled from an alphabet.

### Claim 6 (depends on 1)

The method of claim 1 wherein perturbing the input prompts includes using the randomizing function to sample consecutive characters in each of the input prompts and replacing the sample consecutive characters with characters sampled from an alphabet.

### Claim 7

(canceled)

### Claim 8 (depends on 1)

The method of claim 1 wherein the function assigns a first value to a response that includes the objectionable content and a second value to the response when the response does not include the objectionable content.

### Claim 9 (depends on 8)

The method of claim 8 wherein determining, using the function, that a majority of the responses do not include the objectionable content includes determining that the function assigns the second value to the majority of the responses.

### Claim 10 (depends on 1)

The method of claim 1 comprising setting hyperparameters for controlling the perturbing, the aggregating, and the selecting.

### Claim 11

A system for defending a large language model (LLM) against a jailbreaking attack, the system comprising:
at least one processor and a memory; and
an LLM jailbreaking attack mitigator implemented by the at least one processor for receiving an input prompt for generating output from the LLM, wherein the input prompt is a jailbreaking attack and includes a goal string requesting objectionable content from the LLM and an adversarial portion that when added to the goal string is intended to cause the LLM to output the objectionable content and thereby jailbreak the LLM, generating N versions, N being an integer of at least two, of the input prompt, wherein generating, without knowledge of a position or presence of the adversarial portion of the input prompt, each of the N versions includes perturbing, using a randomizing function, the input prompt to generate N perturbed input prompts, providing the N perturbed input prompts as input to the LLM, which generates N responses, aggregating the responses by estimating a majority effect of the perturbed input prompts on the LLM, wherein estimating the majority effect includes evaluating, for each of the responses, a function determines whether or not the response includes the objectionable content and determining, using the function, that a majority of the responses do not include the objectionable content, and selecting, as output, one of the responses in the majority that does not include the objectionable content, thereby nullifying the jailbreaking attack.

### Claim 12 (depends on 11)

The system of claim 11 wherein the LLM jailbreaking attack mitigator is configured to perturb the input prompts by adding characters to each of the input prompts.

### Claim 13 (depends on 12)

The system of claim 12 wherein the LLM jailbreaking attack mitigator is configured to add the characters to each of the input prompts by using the randomizing function to select characters in each of the input prompts and adding a character sampled from an alphabet of characters after each of the selected characters.

### Claim 14 (depends on 11)

The system of claim 11 wherein the LLM jailbreaking attack mitigator is configured to the input prompts by replacing characters in each of the input prompts.

### Claim 15 (depends on 14)

The system of claim 14 wherein the LLM jailbreaking attack mitigator is configured to replace the characters by using the randomizing function to select character locations in each of the input prompts and replacing characters at the selected locations with characters sampled from an alphabet.

### Claim 16 (depends on 15)

The system of claim 15 wherein the LLM jailbreaking attack mitigator is configured to perturb the input prompts includes by using the randomizing function to sample consecutive characters in each of the input prompts and replacing the sample consecutive characters with characters sampled from an alphabet.

### Claim 17

(canceled)

### Claim 18 (depends on 11)

The system of claim 11 wherein the function assigns a first value to a response that includes the objectionable content and a second value to the response when the response does not include the objectionable content.

### Claim 19 (depends on 18)

The system of claim 18 wherein the LLM jailbreaking attack mitigator is configured to determine, using the function, that a majority of the responses do not include the objectionable content by determining that the function assigns the second value to the majority of the responses.

### Claim 20

A non-transitory computer readable medium having stored thereon executable instructions that when executed by a processor of a computer controls the computer to perform steps comprising:
receiving an input prompt for generating output from a large language model (LLM), wherein the input prompt is a jailbreaking attack and includes a goal string requesting objectionable content from the LLM and an adversarial portion that when added to the goal string is intended to cause the LLM to output the objectionable content and thereby jailbreak the LLM;
generating N versions, N being an integer of at least two, of the input prompt, wherein generating, without knowledge of a position or presence of the adversarial portion of the input prompt, each of the N versions includes perturbing, using a randomizing function, the input prompt to generate N perturbed input prompts;
providing the N perturbed input prompts as input to the LLM, which generates N responses;
aggregating the responses by estimating a majority effect of the perturbed input prompts on the LLM, wherein estimating the majority effect includes evaluating, for each of the responses, a function determines whether or not the response includes the objectionable content and determining, using the function, that a majority of the responses do not include the objectionable content; and
selecting, as output, one of the responses in the majority that does not include the objectionable content, thereby nullifying the jailbreaking attack, thereby nullifying the jailbreaking attack.
