Token optimization through minimized payload for large language models
AssigneeSap SE
InventorDavid Kunz
Life of this patent
Estimating the term…
Loading patent…
AssigneeSap SE
InventorDavid Kunz
Estimating the term…
In an example embodiment, a solution is provided by utilizing an intermediate file format for either an input prompt to or output from a large language model (LLM) (or both). This intermediate file format is one that has the property of being minimizable, meaning that tokens contained in a file of that intermediate file format can be stripped out or otherwise removed without changing the semantic meaning of the file. The input prompt can be created in or converted to this intermediate file format and then minimized prior to being sent to the LLM for text generation. Furthermore, a system message included with the input prompt may instruct the LLM to generate text in the intermediate file format, in minimized form. A specialized parser may then be included to parse the minimized output produced by the LLM.
A system comprising:
A method comprising:
A non-transitory machine-readable medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:
Dependencies are inferred from the claim text.