Mitigation for prompt injection in A.I. models capable of accepting text input
OwnerJonathan Cefalu, Jeremy Charles McHugh +1inventors
InventorsJonathan Cefalu, Jeremy Charles McHugh, Ron Heichman
Life of this patent
Estimating the term…
Abstract
A system for use with an artificial intelligence (AI) model configured to accept text input, such as generative pre-trained transformer (GPT), that detects and tags trusted instructions and nontrusted instructions of an input provided by a user responsive to an AI model prompt. The system uses reinforcement learning (RL) and a set of rules to remove the untrusted instructions from the input and provide only trusted instructions to the AI model. The input is represented as tokens, wherein the trusted instructions and the untrusted instructions are represented using incompatible token sets.
Claims18
A system, comprising:
- an artificial intelligence (AI) model configured to accept electronic text input comprising instructions including a sequence of tokens in response to an AI model prompt;
- and
- a processor configured to:
- receive the instructions including the sequence of tokens;
- determine electronic trusted instructions and electronic untrusted instructions from the electronic text input provided responsive to the AI model prompt;
- electronically tag the electronic trusted instructions with a trusted tag and electronically tag the electronic untrusted instructions with an untrusted tag, the trusted tag and the untrusted tag being represented by token vectors respectively indicating a trusted or untrusted source of the sequence of tokens, wherein the trusted tag and the untrusted tag are respectively indicative of a type of instruction including at least a trusted instruction represented by a first token set adapted to be executed by the AI model to produce trusted human-like text responsive to the trusted instruction and an untrusted instruction represented by a second token set adapted to not be executed by the AI model;
- and
- modify the instructions including the sequence of tokens provided in response to the AI model prompt to indicate that instructions tagged with the trusted tag are to be obeyed and that instructions tagged with the untrusted tag are to be disregarded and to remove instructions tagged with the untrusted tag from the sequence of tokens provided responsive to the AI model prompt to create instructions to the AI model that are influenced by the instructions tagged with the trusted tag but not influenced by the instructions tagged with the untrusted tag,
- wherein the AI model is configured to execute the instructions in the electronic text input that has been tagged with the trusted tag to provide the trusted human-like text.
A method of instructing an artificial intelligence (AI) model configured to accept electronic text input comprising instructions including a sequence of tokens in response to an AI model prompt, the method comprising:
- determining electronic trusted instructions and electronic untrusted instructions from the electronic text input provided responsive to the AI model prompt;
- electronically tagging the electronic trusted instructions with a trusted tag and electronically tagging the electronic untrusted instructions with an untrusted tag, the trusted tag and the untrusted tag being represented by token vectors respectively indicating a trusted or untrusted source of the sequence of tokens, wherein the trusted tag and the untrusted tag are respectively indicative of a type of instruction including at least a trusted instruction represented by a first token set adapted to be executed by the AI model to produce trusted human-like text responsive to the trusted instruction and an untrusted instruction represented by a second token set adapted to not be executed by the AI model;
- and
- modifying the instructions including the sequence of tokens provided in response to the AI model prompt to indicate that instructions tagged with the trusted tag are to be obeyed and that instructions tagged with the untrusted tag are to be disregarded and to remove instructions tagged with the untrusted tag from the sequence of tokens provided responsive to the AI model prompt to create instructions to the AI model that are influenced by the instructions tagged with the trusted tag but not influenced by the instructions tagged with the untrusted tag.
Dependencies are inferred from the claim text.