Text generation methods and apparatuses, storage medium devices, and program products
AssigneeAlipay (Hangzhou) Information Technology Co., Ltd.
InventorsChangxu Shao, Yuhong Guo, Junping Zhao
Life of this patent
Estimating the term…
Loading patent…
AssigneeAlipay (Hangzhou) Information Technology Co., Ltd.
InventorsChangxu Shao, Yuhong Guo, Junping Zhao
Estimating the term…
This specification provides text generation methods, apparatuses, and storage medium devices. One method includes the following operations. In an iteration of a plurality of iterations under a large language model (LLM): estimating a first text sequence following a current text sequence based on a speculative decoding method, forming a plurality of candidate sequences based on the current text sequence and subsequences of the first text sequence, allocating logical blocks to text units in the plurality of candidate sequences in a key-value cache, to store attention information of the text units, mapping the allocated logical blocks to physical blocks based on a first criterion, and determining, by the LLM, a newly generated text unit in the iteration by using attention information of each candidate sequence in the key-value cache, to form a current text sequence for a next iteration of the plurality of iterations.
A method for generating an output text for an input text based on performing a plurality of iterations, the method comprising:
An apparatus for generating an output text for an input text based on performing a plurality of iterations, the apparatus comprising:
A non-transitory, computer-readable medium storing one or more instructions executable by at least one processor to perform operations comprising:
Dependencies are inferred from the claim text.