# US 2026/0065060 A1 · False positive sensitive training of neural networks for malicious prompt classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065060 A1
- Document ID: US20260065060A1
- Status: Published application
- Kind: A1
- Application number: 18/821,447
- Filed: 2024-08-30
- Published: 2026-03-05
- Earliest claimed: 2024-08-30
- Assignees: Palo Alto Networks, Inc.
- Applicants: Palo Alto Networks, Inc.
- Inventors: Brody James Kutt; Samarth Keshari; Yazdan Jamshidikhezeli; Sergey Sviridov
- Web page: https://patentgrep.com/patent/US20260065060A1

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A double cross-entropy loss function is a modification of the standard cross-entropy loss function that is tunable to penalize specific error types, i.e., false positives and false positives for binary classification. A prompt classifier is trained using the double cross-entropy loss function to classify prompts as malicious or benign. The double cross-entropy loss function for the prompt classifier is tuned so that false positive classifications are heavily penalized. The resulting trained prompt classifier maintains a high true positive rate while having a classification threshold that keeps the false positive rate very small. The trained prompt classifier is deployed in a high-load environment for prompt classification.

## Claims

### Claim 1

A method comprising:
training a first composition of a first language model and a first natural language processing model to output confidence values that documents are malicious or benign with a low rate of false positive verdicts obtained from the confidence values, wherein training the first composition comprises, for each first training iteration and corresponding training documents,
invoking the first composition on the training documents to obtain first confidence values that the training documents are malicious or benign; and
backpropagating first loss through the first language model and the first natural language processing model, wherein the first loss quantifies a difference between the first confidence values and ground-truth malicious or benign labels for the training documents, further wherein the first loss is evaluated with a loss function that promotes a low false positive rate for malicious document verdicts obtained based on outputs of the first language model.

### Claim 2 (depends on 1)

The method of claim 1, wherein the loss function comprises a sum of a cross-entropy loss function and a loss function that penalizes different error types in classifications.

### Claim 3 (depends on 1)

The method of claim 1, wherein the loss function comprises the double cross-entropy loss function.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first natural language processing model comprises one or more tokenization layers, one or more embedding layers, and one or more dynamic compression layers.

### Claim 5 (depends on 1)

The method of claim 1, further comprising training a second composition of a second language model and a second natural language processing model, wherein training the second composition comprises, for each second training iteration and corresponding training documents,
invoking the second natural language processing model on the training documents to obtain vector embeddings of the training documents;
invoking the first language model and the second language model on the vector embeddings to obtain second confidence values and third confidence values, respectively, that the training documents are malicious or benign; and
backpropagating second loss through the second language model and the second natural language processing model, wherein the second loss comprises a sum of the loss function evaluated on the third confidence values and a knowledge distillation loss function that takes the second confidence values and the third confidence values as inputs.

### Claim 6 (depends on 5)

The method of claim 5, wherein the second language model comprises a lightweight convolutional neural network.

### Claim 7 (depends on 1)

The method of claim 1, wherein the training documents comprise known malicious or benign prompts to a generative artificial intelligence system.

### Claim 8 (depends on 1)

The method of claim 1, wherein the first language model comprises a transformer neural network.

### Claim 9

A non-transitory, machine-readable medium having program code stored thereon, the program code comprising instructions to:
train a first classifier to output probabilities that documents are malicious or benign with a low rate of false positive verdicts obtained from the probabilities, wherein the first classifier comprises a composition of a first natural language processing model and a first neural network, wherein instructions to train the first neural network comprise instructions to, for each first training iteration and corresponding training documents,
invoke the first classifier on the training documents to obtain first probabilities that the training documents are malicious or benign; and
backpropagate first loss through the first classifier, wherein the first loss quantifies a difference between the first probabilities and ground-truth malicious or benign labels for the training documents, further wherein the first loss is evaluated with a loss function that promotes a low false positive rate for malicious document verdicts obtained based on outputs of the first classifier.

### Claim 10 (depends on 9)

The machine-readable medium of claim 9, wherein the loss function comprises a sum of a cross-entropy loss function and a loss function that penalizes different error types in classifications.

### Claim 11 (depends on 9)

The machine-readable medium of claim 9, wherein the loss function comprises the double cross-entropy loss function.

### Claim 12 (depends on 9)

The machine-readable medium of claim 9, wherein the first natural language processing model comprises one or more tokenization layers, one or more embedding layers, and one or more dynamic compression layers.

### Claim 13 (depends on 9)

The machine-readable medium of claim 9, wherein the program code further comprises instructions to train a second classifier, wherein the second classifier comprises composition of a second natural language processing model and a second neural network, wherein the instructions to train the second classifier comprise instructions to, for each second training iteration and corresponding training documents,
invoke the first classifier and the second classifier on the documents to obtain second probabilities and third probabilities, respectively, that the training documents are malicious or benign; and
backpropagate second loss through the second classifier, wherein the second loss comprises a sum of the loss function evaluated on the third probabilities and a knowledge distillation loss function that takes the second probabilities and the third probabilities as inputs.

### Claim 14 (depends on 13)

The machine-readable medium of claim 13, wherein the second classifier comprises a lightweight convolutional neural network.

### Claim 15 (depends on 9)

The machine-readable medium of claim 9, wherein the training documents comprise known malicious or benign prompts to a generative artificial intelligence system.

### Claim 16 (depends on 9)

The machine-readable medium of claim 9, wherein the first classifier comprises a transformer neural network.

### Claim 17 (depends on 9)

The machine-readable medium of claim 9, wherein the instructions to backpropagate the first loss through the first classifier comprise instructions to backpropagate the first loss through the first neural network and one or more layers of the first natural language processing model.

### Claim 18

An apparatus comprising:
a processor; and
a machine-readable medium having instructions stored thereon that are executable by the processor to cause the apparatus to,
train a first classifier to output scores that documents are malicious or benign with a low rate of false positive verdicts obtained from the scores, wherein the first classifier comprises a composition of a first natural language processing model and a first neural network, wherein the instructions to train the first neural network comprise instructions executable by the processor to cause the apparatus to, for each first training iteration and corresponding training documents,
invoke the first classifier on the training documents to obtain first scores that the training documents are malicious or benign; and
backpropagate first loss through the first classifier, wherein the first loss quantifies a difference between the first scores and ground-truth malicious or benign labels for the training documents, further wherein the first loss is evaluated with a loss function that promotes a low false positive rate for malicious document verdicts obtained based on outputs of the first classifier.

### Claim 19 (depends on 18)

The apparatus of claim 18, wherein the loss function comprises a sum of a cross-entropy loss function and a loss function that penalizes different error types in classifications.

### Claim 20 (depends on 18)

The apparatus of claim 18, wherein the loss function comprises the double cross-entropy loss function.
