Request segmentation for reduced memory consumption by trained sequential models

A system for reducing memory consumption of a trained sequential model comprising:

• a request segmentation engine stored in memory and executable to:
• receive, from a client application, an initial processing request identifying an input sequence for processing by a trained sequential model and an initial value for an output size parameter, the output size parameter representing a requested size of output from the trained sequential model;
• transmit, to the trained sequential model, multiple partial processing requests based on the initial processing request, each of the multiple partial processing requests specifying a fraction of the initial value as the output size parameter;
• receiving a sequence of output responses from the trained sequential model, each response in the sequence of output responses being generated in response to processing of one of the multiple partial processing requests;
• and
• returning, to the client application, a final merged response that includes the sequence of output responses formatted to match expected response output associated with the initial processing request.

One or more tangible computer-readable storage media encoding computer-executable instructions for executing a computer process for reducing resource consumption of a trained sequential model, the computer process comprising:

• receiving, from a client application, an initial processing request identifying an input sequence to be processed by the trained sequential model and an initial value for an output size parameter, the output size parameter representing a requested size of output from the trained sequential model;
• transmitting, to the trained sequential model, multiple partial processing requests based on the initial processing request that each specify a fraction of the initial value as the output size parameter;
• receiving a sequence of output responses from the trained sequential model, each response in the sequence of output responses being generated in response to processing of one of the multiple partial processing requests;
• and
• returning, to the client application, a final merged response that includes the sequence of output responses formatted to match expected response output associated with the initial processing request.