# US 2026/0094028 A1 · Method for performing acceleration procedure to accelerate inference procedure of large language model > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2026/0094028 A1 - Document ID: US20260094028A1 - Status: Published application - Kind: A1 - Application number: 19/340,885 - Filed: 2025-09-26 - Published: 2026-04-02 - Earliest claimed: 2024-09-27 - Assignees: Mediatek Inc. - Applicants: Mediatek Inc. - Inventors: Yue-Ting Pan; Huai-Ting Li; Yi-Min Tsai; Ya-Lin Huang; I-Lin Chen - Web page: https://patentgrep.com/patent/US20260094028A1 ## CPC - G06N 5/04: Inference or reasoning models - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for performing an acceleration procedure to accelerate an inference procedure of a large language model (LLM) includes: performing a first drafting procedure to generate multiple first draft tokens; according to first draft information related to the multiple first draft tokens, determining whether a first rule is met to generate a first determination result, wherein the first rule corresponds to the first acceleration procedure; in response to the first determination result indicating that the first rule is not met, performing a second drafting procedure to generate multiple second draft tokens; obtaining multiple formal draft tokens at least based on the multiple second draft tokens; inputting the multiple formal draft tokens to the LLM in order to generate multiple target tokens; and performing a matching operation upon the multiple formal draft tokens and the multiple target tokens to generate at least one output tokens of the LLM. ## Claims ### Claim 1 A method for performing an acceleration procedure to accelerate an inference procedure of a large language model (LLM) comprising: performing a first drafting procedure to generate multiple first draft tokens; according to first draft information related to the multiple first draft tokens, determining whether a first rule is met in order to generate a first determination result, wherein the first rule corresponds to the first acceleration procedure; in response to the first determination result indicating that the first rule is not met, performing a second drafting procedure to generate multiple second draft tokens; obtaining multiple formal draft tokens at least based on the multiple second draft tokens; inputting the multiple formal draft tokens to the LLM in order to generate multiple target tokens; and performing a matching operation upon the multiple formal draft tokens and the multiple target tokens to generate at least one output token of the LLM. ### Claim 2 (depends on 1) The method of claim 1, further comprising: in response to the first determination result indicating that the first rule is met, obtaining the multiple formal draft tokens based on the multiple first draft tokens. ### Claim 3 (depends on 2) The method of claim 2, wherein the step of obtaining the multiple formal draft tokens based on the multiple first draft tokens comprises: utilizing the multiple first draft tokens as the multiple formal draft tokens. ### Claim 4 (depends on 1) The method of claim 1, wherein the first drafting procedure is a retrieval-based drafting procedure, and the second drafting procedure is a drafter-based drafting procedure. ### Claim 5 (depends on 1) The method of claim 1, wherein the first draft information comprises a window size value corresponding to the multiple first draft tokens retrieved from stored data; and the first rule is related to a relationship between the window size value and a threshold value. ### Claim 6 (depends on 5) The method of claim 5, wherein in response to the window size value being greater than or equal to the threshold value, the first determination result indicates that the first rule is met. ### Claim 7 (depends on 1) The method of claim 1, wherein the step of obtaining the multiple formal draft tokens at least based on the multiple second draft tokens comprises: utilizing the multiple second draft tokens as the multiple formal draft tokens. ### Claim 8 (depends on 1) The method of claim 1, wherein the step of obtaining the multiple formal draft tokens at least based on the multiple second draft tokens comprises: performing a fusion operation upon the multiple first draft tokens and the multiple second draft tokens to generate the multiple formal draft tokens. ### Claim 9 (depends on 1) The method of claim 1, further comprising: according to second draft information related to the multiple second draft tokens, determining whether a second rule is met in order to generate a second determination result, wherein the second rule corresponds to the second drafting procedure; wherein the multiple formal draft tokens are obtained at least based on the multiple second draft tokens in response to the second determination result indicating that the second rule is met. ### Claim 10 (depends on 9) The method of claim 9, further comprising: in response to the second determination result indicating that the second rule is not met, performing a third drafting procedure to generate multiple third draft tokens; and obtaining the multiple formal draft tokens at least based on the multiple third draft tokens. ### Claim 11 A non-transitory machine-readable medium for storing a program code, wherein when loaded and executed by a processor, the program code instructs the processor to perform a method for performing an acceleration procedure to accelerate an inference procedure of a large language model (LLM); and the method comprises: performing a first drafting procedure to generate multiple first draft tokens; according to first draft information related to the multiple first draft tokens, determining whether a first rule is met in order to generate a first determination result, wherein the first rule corresponds to the first acceleration procedure; in response to the first determination result indicating that the first rule is not met, performing a second drafting procedure to generate multiple second draft tokens; obtaining multiple formal draft tokens at least based on the multiple second draft tokens; inputting the multiple formal draft tokens to the LLM in order to generate multiple target tokens; and performing a matching operation upon the multiple formal draft tokens and the multiple target tokens to generate at least one output token of the LLM. ### Claim 12 (depends on 11) The non-transitory machine-readable medium of claim 11, wherein the method further comprises: in response to the first determination result indicating that the first rule is met, obtaining the multiple formal draft tokens based on the multiple first draft tokens. ### Claim 13 (depends on 12) The non-transitory machine-readable medium of claim 12, wherein the step of obtaining the multiple formal draft tokens based on the multiple first draft tokens comprises: utilizing the multiple first draft tokens as the multiple formal draft tokens. ### Claim 14 (depends on 11) The non-transitory machine-readable medium of claim 11, wherein the first drafting procedure is a retrieval-based drafting procedure, and the second drafting procedure is a drafter-based drafting procedure. ### Claim 15 (depends on 11) The non-transitory machine-readable medium of claim 11, wherein the first draft information comprises a window size value corresponding to the multiple first draft tokens retrieved from stored data; and the first rule is related to a relationship between the window size value and a threshold value. ### Claim 16 (depends on 15) The non-transitory machine-readable medium of claim 15, wherein in response to the window size value being greater than or equal to the threshold value, the first determination result indicates that the first rule is met. ### Claim 17 (depends on 11) The non-transitory machine-readable medium of claim 11, wherein the step of obtaining the multiple formal draft tokens at least based on the multiple second draft tokens comprises: utilizing the multiple second draft tokens as the multiple formal draft tokens. ### Claim 18 (depends on 11) The non-transitory machine-readable medium of claim 11, wherein the step of obtaining the multiple formal draft tokens at least based on the multiple second draft tokens comprises: performing a fusion operation upon the multiple first draft tokens and the multiple second draft tokens to generate the multiple formal draft tokens. ### Claim 19 (depends on 11) The non-transitory machine-readable medium of claim 11, wherein the method further comprises: according to second draft information related to the multiple second draft tokens, determining whether a second rule is met in order to generate a second determination result, wherein the second rule corresponds to the second drafting procedure; wherein the multiple formal draft tokens are obtained at least based on the multiple second draft tokens in response to the second determination result indicating that the second rule is met. ### Claim 20 (depends on 19) The non-transitory machine-readable medium of claim 19, wherein the method further comprises: in response to the second determination result indicating that the second rule is not met, performing a third drafting procedure in order to generate multiple third draft tokens; and obtaining the multiple formal draft tokens at least based on the multiple third draft tokens.