Processing requests using compressed and complete machine learning models

A method, comprising:

• performing, by a host of a machine learning-based service of a provider network:
• receiving a request to process data, wherein the request indicates the data to be processed;
• in response to receiving the request to process the data, processing an initial portion of the data using a compressed version of a model that corresponds to a complete version of the model to generate one or more results and begin loading, by the host of the provider network, of the complete version of the model into the memory of the host of the provider network;
• returning the one or more results;
• complete the loading of the complete version of the model into a memory of the host;
• in response to a determination, by the host, that the complete version of the model is loaded into the memory of the host, process a remaining portion of the data indicated by the same request to process the data using the complete version of the model instead of the compressed version of the model that was used to process the initial portion of the data indicated by the same request to process data to generate one or more other results;
• and
• returning the one or more other results.

One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors cause the one or more processors to:

• perform, by a host of a machine learning-based service of a provider network:
• receiving a request to process data, wherein the request indicates the data to be processed;
• in response to receiving the request to process the data, process an initial portion of the data using a compressed version of a model that corresponds to a complete version of the model to generate one or more results and begin loading, by the host of the provider network, of the complete version of the model into the memory of the host of the provider network;
• return the one or more results;
• complete the loading of the complete version of the model into a memory of the host;
• in response to a determination, by the host, that the complete version of the model is loaded into the memory of the host, process a remaining portion of the data indicated by the same request to process the data using the complete version of the model instead of the compressed version of the model that was used to process the initial portion of the data indicated by the same request to process data to generate one or more other results;
• and
• return the one or more other results.