Domain-aware large language model governance
AssigneeSalesforce, Inc.
InventorsSarath Shekkizhar, Adam Earle
Life of this patent
Estimating the term…
Abstract
Techniques are described herein for decreasing the likelihood of out-of-domain LLM responses. In operation, it is determined whether a representation of an input token, generated during an intermediate step of the LLM operations, is in-domain or out-of-domain based on a previously detected sub-space. In addition, the response being generated by the LLM is constrained when it is determined the response being generated is not within the previously detected sub-space.
Claims21
A method to constrain responses being generated by a large language model (LLM), the method comprising:
- generating, by the LLM being executed on a processing device, a response to an input, wherein the generating includes:
- determining, by a transformer block within a plurality of transformer blocks of the LLM, an output that includes a representation of at least one of a plurality of tokens;
- determining, based at least on the output and a previously detected sub-space representing outputs of the transformer block that have a characteristic, whether the response being generated is not within the previously detected sub-space, wherein the characteristic represents one or more portions of a specific domain or alignment with a policy;
- constraining the response being generated by the LLM when it is determined the response being generated is not within the previously detected sub-space;
- and
- causing the response to be presented to a user.
A non-transitory computer-readable storage medium storing executable instructions, which when executed by a processing device, are configurable to cause the processing device to perform operations comprising:
- generating, by a larger language model (LLM) being executed on the processing device, a response to an input, wherein the generating includes:
- determining, by a transformer block within a plurality of transformer blocks of the LLM, an output that includes a representation of at least one of a plurality of tokens;
- determining, based at least on the output and a previously detected sub-space representing outputs of the transformer block that have a characteristic, whether the response being generated is not within the previously detected sub-space, wherein the characteristic represents one or more portions of a specific domain or alignment with a policy;
- constraining the response being generated by the LLM when it is determined the response being generated is not within the previously detected sub-space;
- and
- causing the response to be presented to a user.
A system to constrain responses being generated by a large language model (LLM), the system comprising:
- a set of one or more processors;
- and
- a non-transitory machine-readable storage medium that provides instructions that, when executed by one or any combination of the set of processors, are configurable to cause the system to perform operations comprising:
- generating, by the LLM, a response to an input, wherein the generating includes:
- determining, by a transformer block within a plurality of transformer blocks of the LLM, an output that includes a representation of at least one of a plurality of tokens;
- determining, based at least on the output and a previously detected sub-space representing outputs of the transformer block that have a characteristic, whether the response being generated is not within the previously detected sub-space, wherein the characteristic represents one or more portions of a specific domain or alignment with a policy;
- constraining the response being generated by the LLM when it is determined the response being generated is not within the previously detected sub-space;
- and
- causing the response to be presented to a user.
Dependencies are inferred from the claim text.