Domain-aware large language model governance
AssigneeSalesforce, Inc.
InventorsSarath Shekkizhar, Adam Earle
Life of this patent
Estimating the term…
Abstract
Techniques are described herein for a method of decreasing the likelihood of out-of-domain LLM responses. The method includes determining, by a block of a LLM, a representation of the text input. The method further includes determining a set of coefficients based at least on a reconstruction of the text input using a dictionary and the representation of the text input. The method further includes performing a sparsity check using the set of coefficients. The method further includes generating a response to the text input based at least on the sparsity check.
Claims26
A method to constrain responses being generated by a large language model (LLM), the method comprising:
- generating, by a prompt manager, an input for the LLM, wherein the input comprises a plurality of tokens;
- generating, by the LLM being executed on a processing device, a response to the input, wherein the generating includes:
- determining, by a block of the LLM, an output that includes a representation of at least one of the plurality of tokens;
- determining a set of coefficients based at least on the output and a template predictor, wherein the template predictor was previously determined to predict templates in a dictionary responsive to outputs from the block, wherein the dictionary defines a previously detected sub-space representing outputs of the block that have a characteristic, wherein the characteristic represents one or more portions of a specific domain or alignment with a policy;
- performing a sparsity check using the set of coefficients;
- and
- constraining the response being generated by the LLM when it is determined the response being generated is not within the previously detected subspace based at least in part on the sparsity check;
- and
- causing the response to be presented to a user as at least part of a subsequent turn in a conversation in which the user is actively engaged.
A non-transitory computer-readable storage medium storing executable instructions, which when executed by a processing device, cause the processing device to perform operations comprising:
- generating, by a prompt manager, an input for a large language model (LLM), wherein the input comprises a plurality of tokens;
- generating, by the LLM, a response to the input, wherein the generating includes:
- determining, by a block of the LLM, an output that includes a representation of at least one of the plurality of tokens;
- determining a set of coefficients based at least on the output and a template predictor, wherein the template predictor was previously determined to predict templates in a dictionary responsive to outputs from the block, wherein the dictionary defines a previously detected sub-space representing outputs of the block that have a characteristic, wherein the characteristic represents one or more portions of a specific domain or alignment with a policy;
- performing a sparsity check using the set of coefficients;
- and
- constraining the response being generated by the LLM when it is determined the response being generated is not within the previously detected subspace based at least in part on the sparsity check;
- and
- causing the response to be presented to a user as at least part of a subsequent turn in a conversation in which the user is actively engaged.
A system comprising:
- a memory component;
- and
- a processing device coupled to the memory component, the processing device to perform operations comprising:
- generating, by a prompt manager, and input for a large language model (LLM), wherein the input comprises a plurality of tokens;
- generating, by the LLM, a response to the input, wherein the generating includes:
- determining, by a block of the LLM, an output that includes a representation of at least one of the plurality of tokens;
- determining a set of coefficients based at least on the output and a template predictor, wherein the template predictor was previously determined to predict templates in a dictionary responsive to outputs from the block, wherein the dictionary defines a previously detected sub-space representing outputs of the block that have a characteristic, wherein the characteristic represents one or more portions of a specific domain or alignment with a policy;
- performing a sparsity check using the set of coefficients;
- and
- constraining the response being generated by the LLM when it is determined the response being generated is not within the previously detected subspace based at least in part on the sparsity check;
- and
- causing the response to be presented to a user as at least part of a subsequent turn in a conversation in which the user is actively engaged.
Dependencies are inferred from the claim text.