US 2026/0228331 A1Published application

Malicious ai prompt detection

AssigneeSap SE

InventorsShubham SAKLANI, Prashant TELKAR, Meldon Malcolm DCUNHA, Vishwas AGRAWAL, Ankit SHARMA

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system and method include reception of a prompt, determination that the prompt is not semantically similar to any of a plurality of text generation model prompts, in response to the determination, prompt a first text generation model to determine whether the prompt is malicious, in response to a determination that the prompt is not malicious, prompt a second text generation model with the prompt to determine a first prompt output, prompt a third text generation model to determine whether the first prompt output is malicious, and, in response a the determination that the third text generation model determined that the first prompt output is not malicious, return the prompt output in response to the prompt.

Claims20

Dependent claims

A system comprising:

a memory storing program code;
and
at least one processing unit to execute the program code to cause the system to:
receive a first text generation model prompt;
determine that the first text generation model prompt is not semantically similar to any of a plurality of text generation model prompts;
in response to the determination that the first text generation model prompt is not semantically similar to any of the plurality of text generation model prompts, prompt a first text generation model to determine whether the first text generation model prompt is malicious;
determine that the first text generation model determined that the first text generation model prompt is not malicious;
in response to the determination that the first text generation model determined that the first text generation model prompt is not malicious, prompt a second text generation model with the first text generation model prompt to determine a first prompt output;
prompt a third text generation model to determine whether the first prompt output is malicious;
determine that the third text generation model determined that the first prompt output is not malicious;
and
in response to the determination that the third text generation model determined that the first prompt output is not malicious, return the prompt output in response to the first text generation model prompt.

9 more elements

A method comprising:

receiving a first text generation model prompt;
determining that the first text generation model prompt is not semantically similar to any of a plurality of text generation model prompts;
in response to determining that the first text generation model prompt is not semantically similar to any of the plurality of text generation model prompts, prompt a first text generation model to determine whether the first text generation model prompt is malicious;
determining that the first text generation model determined that the first text generation model prompt is not malicious;
in response to determining that the first text generation model determined that the first text generation model prompt is not malicious, prompt a second text generation model with the first text generation model prompt to determine a first prompt output;
prompting a third text generation model to determine whether the first prompt output is malicious;
determining that the third text generation model determined that the first prompt output is not malicious;
and
in response to determining that the third text generation model determined that the first prompt output is not malicious, return the prompt output in response to the first text generation model prompt.

6 more elements

One or more non-transitory computer-readable recording media storing program code, the program code executable by at least one processing unit of a computing system to cause the computing system to perform operations comprising:

receiving a first text generation model prompt;
determining that the first text generation model prompt is not semantically similar to any of a plurality of text generation model prompts;
in response to determining that the first text generation model prompt is not semantically similar to any of the plurality of text generation model prompts, prompt a first text generation model to determine whether the first text generation model prompt is malicious;
determining that the first text generation model determined that the first text generation model prompt is not malicious;
in response to determining that the first text generation model determined that the first text generation model prompt is not malicious, prompt a second text generation model with the first text generation model prompt to determine a first prompt output;
prompting a third text generation model to determine whether the first prompt output is malicious;
determining that the third text generation model determined that the first prompt output is not malicious;
and
in response to determining that the third text generation model determined that the first prompt output is not malicious, return the prompt output in response to the first text generation model prompt.

6 more elements

Dependencies are inferred from the claim text.