# US 2025/0094233 A1 · Request segmentation for reduced memory consumption by trained sequential models > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0094233 A1 - Document ID: US20250094233A1 - Status: Published application - Kind: A1 - Application number: 18/470,827 - Filed: 2023-09-20 - Published: 2025-03-20 - Earliest claimed: 2023-09-20 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Wenbin MENG; Hemant KUMAR; Rakesh KELKAR; Karthik RAMAN; Sanjay RAMANUJAN; Kevin Joseph RIEHM; Theodore Dragov TODOROV - Web page: https://patentgrep.com/patent/US20250094233A1 ## CPC - G06F 9/5044: {considering hardware capabilities} - G06F 16/3329: Natural language query formulation - G06F 9/5083: {Techniques for rebalancing the load in a distributed system} - G06N 3/045: Combinations of networks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A disclosed method reduces memory consumption of a trained sequential model. The method includes receiving, from a client application, an initial processing request identifying an input sequence to be processed by the trained sequential model and an initial value for an output size parameter specifying a requested size of output from the trained sequential model. The method further includes sequentially transmitting, to the trained sequential model, multiple partial processing requests based on the initial processing request that each specify a fraction of the initial value as the output size parameter and receiving a sequence of output responses from the trained sequential model generated in response to processing the multiple partial processing requests. The method further provides for returning, to the client application, a final merged response that includes the sequence of output responses. ## Claims ### Claim 1 A method for reducing memory consumption of a trained sequential model, the method comprising: receiving, from a client application, an initial processing request identifying an input sequence to be processed by the trained sequential model and an initial value for an output size parameter, the output size parameter representing a requested size of output from the trained sequential model; sequentially transmitting, to the trained sequential model, multiple partial processing requests based on the initial processing request that each specify a fraction of the initial value as the output size parameter; receiving a sequence of output responses from the trained sequential model, each response in the sequence of output responses being generated in response to processing of a corresponding one of the multiple partial processing requests; and returning, to the client application, a final merged response including the sequence of output responses formatted to match expected response output associated with the initial processing request. ### Claim 2 (depends on 1) The method of claim 1, wherein the fraction of the initial value specified for the output size parameter in each different one of the multiple partial processing requests collectively sum to the initial value. ### Claim 3 (depends on 1) The method of claim 1, wherein each of the multiple partial processing requests further includes the input sequence. ### Claim 4 (depends on 1) The method of claim 1, wherein the multiple partial processing requests include a first partial processing request and one or more additional partial processing requests, each of the one or more additional partial processing requests identifying a subset of the sequence of output responses generated during processing of previous requests of the multiple partial processing requests. ### Claim 5 (depends on 1) The method of claim 1, wherein transmitting, to the trained sequential model, multiple partial processing requests includes: transmitting, to the trained sequential model, a first partial processing request that includes the input sequence; receiving a first processing result in response to processing of the first partial processing request, the first processing result including a first output sequence corresponding to a first sequential portion of the final merged response; transmitting, to the trained sequential model, a second partial processing request that includes the input sequence and the first output sequence; and receiving, from the trained sequential model, a second processing result in response processing of the second partial processing request, the second processing result including the first output sequence and a second output sequence corresponding to a second sequential portion of the final merged response, wherein the final merged response includes the second output sequence appended to the first output sequence. ### Claim 6 (depends on 1) The method of claim 1, wherein the trained sequential model is a generative transformer-based model. ### Claim 7 (depends on 1) The method of claim 1, wherein transmitting of the multiple partial processing requests is performed by a client device executing the client application. ### Claim 8 A system for reducing memory consumption of a trained sequential model comprising: a request segmentation engine stored in memory and executable to: receive, from a client application, an initial processing request identifying an input sequence for processing by a trained sequential model and an initial value for an output size parameter, the output size parameter representing a requested size of output from the trained sequential model; transmit, to the trained sequential model, multiple partial processing requests based on the initial processing request, each of the multiple partial processing requests specifying a fraction of the initial value as the output size parameter; receiving a sequence of output responses from the trained sequential model, each response in the sequence of output responses being generated in response to processing of one of the multiple partial processing requests; and returning, to the client application, a final merged response that includes the sequence of output responses formatted to match expected response output associated with the initial processing request. ### Claim 9 (depends on 8) The system of claim 8, wherein the fraction of the initial value specified for the output size parameter in each different one of the multiple partial processing requests collectively sum to the initial value. ### Claim 10 (depends on 8) The system of claim 8, wherein each of the multiple partial processing requests further includes the input sequence. ### Claim 11 (depends on 10) The system of claim 10, wherein the multiple partial processing requests include a first partial processing request and one or more additional partial processing requests, each of the one or more additional partial processing requests including a subset of the sequence of output responses generated during processing of previous requests of the multiple partial processing requests. ### Claim 12 (depends on 10) The system of claim 10, wherein the multiple partial processing requests further include a first partial processing request that includes the input sequence and a second partial processing request that includes the input sequence appended to a first output sequence generated by the trained sequential model based on the first partial processing request. ### Claim 13 (depends on 12) The system of claim 12, wherein the sequence of output responses includes the first output sequence and a second output sequence generated by the trained sequential model based on the second partial processing request. ### Claim 14 (depends on 8) The system of claim 8, wherein the trained sequential model is a transformer model with a decoder-only architecture. ### Claim 15 (depends on 8) The system of claim 8, wherein the request segmentation engine is executed on a same a client device that executes the client application. ### Claim 16 One or more tangible computer-readable storage media encoding computer-executable instructions for executing a computer process for reducing resource consumption of a trained sequential model, the computer process comprising: receiving, from a client application, an initial processing request identifying an input sequence to be processed by the trained sequential model and an initial value for an output size parameter, the output size parameter representing a requested size of output from the trained sequential model; transmitting, to the trained sequential model, multiple partial processing requests based on the initial processing request that each specify a fraction of the initial value as the output size parameter; receiving a sequence of output responses from the trained sequential model, each response in the sequence of output responses being generated in response to processing of one of the multiple partial processing requests; and returning, to the client application, a final merged response that includes the sequence of output responses formatted to match expected response output associated with the initial processing request. ### Claim 17 (depends on 16) The one or more tangible computer-readable storage media of claim 16, wherein the fraction of the initial value specified for the output size parameter in each different one of the multiple partial processing requests collectively sum to the initial value. ### Claim 18 (depends on 16) The one or more tangible computer-readable storage media of claim 16, wherein each of the multiple partial processing requests further includes the input sequence. ### Claim 19 (depends on 16) The one or more tangible computer-readable storage media of claim 16, wherein the multiple partial processing requests include a first partial processing request and one or more additional partial processing requests, each of the one or more additional partial processing requests identifying a subset of the sequence of output responses generated during processing of previous requests of the multiple partial processing requests. ### Claim 20 (depends on 16) The one or more tangible computer-readable storage media of claim 16, wherein transmitting, to the trained sequential model, multiple partial processing requests includes: transmitting, to the trained sequential model, a first partial processing request that includes the input sequence; receiving a first processing result in response to processing of the first partial processing request, the first processing result including a first output sequence corresponding to a first sequential portion of the final merged response; transmitting, to the trained sequential model, a second partial processing request that includes the input sequence and the first output sequence; and receiving, from the trained sequential model, a second processing result in response processing of the second partial processing request, the second processing result including the first output sequence and a second output sequence corresponding to a second sequential portion of the final merged response, wherein the final merged response includes the second output sequence appended to the first output sequence.