Large Language Model Response to Jailbreaks with Self-Correction and Correction with External Feedback
AssigneeOracle International Corporation
InventorsSwetasudha Panda, Michael Louis Wick, Jason Anthony Peck, Naveen Jafer Nizar
Life of this patent
Estimating the term…
Abstract
Systems, methods, and apparatus to implement techniques to correct for jailbreak prompts input to generative language models are described. A prompt is generated that includes a jailbreak prompt, an original response to the jailbreak prompt, and a correction response. The generated response is then submitted to a generative language model and response returned.
Claims20
A system, comprising:
- at least one processor;
- a memory, comprising program instructions that when executed by the at least one processor cause the at least one processor to implement a machine learning model development system, the machine learning model development system configured to:
- for a generative language model under test:
- identify a jailbreak defense technique that comprises generating a prompt to an evaluation model for the generative language model under test, wherein the prompt includes:
- a jailbreak prompt;
- an original language model response;
- and
- a corrective prompt;
- and
- create a jailbreak test pipeline that includes the generative language model under test and the jailbreak defense technique;
- execute the jailbreak test pipeline using a plurality of different test inputs including at least one jailbreak attempt;
- capture one or more performance metrics of the jailbreak technique during execution of the jailbreak test pipeline, wherein the one or more performance metrics comprise a least one over-refusal metric of the generative language model under test;
- and
- providing, via an interface of the machine learning model development system, the one or more performance metrics of the jailbreak technique.
A computer-implemented method, comprising:
- for a generative language model under test:
- identifying a jailbreak defense technique that comprises generating a prompt to an evaluation model for the generative language model under test, wherein the prompt includes:
- a jailbreak prompt;
- an original language model response;
- and
- a corrective prompt;
- and
- creating a jailbreak test pipeline that includes the generative language model under test and the jailbreak defense technique;
- executing the jailbreak test pipeline using a plurality of different test inputs including at least one jailbreak attempt;
- capturing one or more performance metrics of the jailbreak technique during execution of the jailbreak test pipeline, wherein the one or more performance metrics comprise at least one over-refusal metric of the generative language model under test;
- and
- providing, via an interface of machine learning model development system, the one or more performance metrics of the jailbreak technique.
One or more non-transitory, computer-readable storage media, storing program instructions that when executed on or across one or more computing devices, cause the one or more computing devices to implement:
- for a generative language model under test:
- identifying a jailbreak defense technique that comprises generating a prompt to an evaluation model for the generative language model under test, wherein the prompt includes:
- a jailbreak prompt;
- an original language model response;
- and
- a corrective prompt;
- and
- creating a jailbreak test pipeline that includes the generative language model under test and the jailbreak defense technique;
- executing the jailbreak test pipeline using a plurality of different test inputs including at least one jailbreak attempt;
- capturing one or more performance metrics of the jailbreak technique during execution of the jailbreak test pipeline, wherein the one or more performance metrics comprise at least one over-refusal metric of the generative language model under test;
- and
- providing, via an interface of machine learning model development system, the one or more performance metrics of the jailbreak technique.
Dependencies are inferred from the claim text.