US 2026/0310941 A1 · EFFICIENT AUTOREGRESSIVE INFERENCE OF LARGE LANGUAGE MODELS (LLMS) WITH STATIC COMPILATION

Abstract

A processor-implemented method for adapting autoregressive inference of large language models (LLMs) for static compilation includes defining a maximally sized static data structure and a variable data mask. The variable data mask indicates a valid section of the maximally sized data structure. A set of valid data is determined by applying the variable data mask to the maximally sized static data structure. A computation is performed using only the set of valid data.

Claims19

Dependent claims

An apparatus comprising:

at least one memory;
and
at least one processor coupled to the at least one memory, the at least one processor configured to:
define a maximally sized static data structure and a variable data mask, the variable data mask indicating a valid section of the maximally sized static data structure;
determine a set of valid data by applying the variable data mask to the maximally sized static data structure;
and
perform a computation using only the set of valid data.

4 more elements

Dependencies are inferred from the claim text.

Efficient autoregressive inference of large language models (llms) with static compilation

Life of this patent

Efficient autoregressive inference of large language models (llms) with static compilation

Life of this patent

Abstract

Claims19