Back to results
US 2026/0065098 A1Published application
Machine-learned speculative decoding engines
AssigneeGroq, Inc.
InventorAndrew Esper Bitar
Life of this patent
Estimating the term…
Abstract
The present disclosure provides systems and methods for: obtaining a request; obtaining, from a draft model, a first plurality of draft tokens based on the request; determining an error rate associated with the first plurality of draft tokens; determining a modified context length for the draft model based on the error rate; and configuring the draft model based on the modified context length.
Claims20
A method, comprising:
- obtaining a request;
- obtaining, from a draft model, a first plurality of draft tokens based on the request;
- determining an error rate associated with the first plurality of draft tokens;
- determining a modified context length for the draft model based on the error rate;
- and
- configuring the draft model based on the modified context length.
3 more elements
A system, comprising:
- one or more processors;
- and
- one or more non-transitory, computer-readable media storing instructions that, when implemented, cause the one or more processors to perform operations, the operations comprising:
- obtaining a request;
- obtaining, from a draft model, a first plurality of draft tokens based on the request;
- determining an error rate associated with the first plurality of draft tokens;
- determining a modified context length for the draft model based on the error rate;
- and
- configuring the draft model based on the modified context length.
6 more elements
One or more non-transitory, computer-readable media storing instructions that, when implemented, cause one or more processors to perform operations, the operations comprising:
- obtaining a request;
- obtaining, from a draft model, a first plurality of draft tokens based on the request;
- determining an error rate associated with the first plurality of draft tokens;
- determining a modified context length for the draft model based on the error rate;
- and
- configuring the draft model based on the modified context length.
3 more elements
Dependencies are inferred from the claim text.