Distributed data processing and machine learning workload scaling

A system for streamlining a deployment and scaling of data processing and machine learning workloads on a distributed system, the system comprising:

• at least one processor;
• and
• a memory including instructions that, when executed by the at least one processor, cause the at least one processor to perform operations to:
• receive from a user at a user interface:
• obtain input data by querying an input measure group using a filtered query;
• slice the obtained input data into one or more slices based on a slicing key;
• assign at least one slice of the one or more slices for each container of one or more containers to be used;
• execute at each container of the one or more containers the respective assigned at least one slice of the one or more slices using a driver code, a code package, and a virtual environment;
• and
• in response to execution at each container of the respective assigned at least one slice of the one or more slices, generate an output for each input.

At least one non-transitory machine-readable medium including instructions for streamlining a deployment and scaling of data processing and machine learning workloads on a distributed system that, when executed by at least one processor, cause the at least one processor to perform operations to:

• receive from a user at a user interface:
• obtain input data by querying an input measure group using a filtered query;
• slice the obtained input data into one or more slices based on a slicing key;
• assign at least one slice of the one or more slices for each container of one or more containers to be used;
• execute at each container of the one or more containers the respective assigned at least one slice of the one or more slices using a driver code, a code package, and a virtual environment;
• and
• in response to execution at each container of the respective assigned at least one slice of the one or more slices, generate an output for each input.