Providing visibility and recommendations for batch processing

A system comprising:

• one or more processors;
• and
• one or more computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to execute operations comprising:
• causing execution of a batch job on a plurality of virtual compute instances associated with a batch-processing service according to a first scheduling policy of a user account associated with the batch job;
• receiving telemetry data associated with the execution of the batch job on the plurality of virtual compute instances, wherein the telemetry data is based on amounts of computing resources utilized by the batch job during the execution;
• generating a visualization that includes a graphical representation of a plurality of execution metrics over time for the batch job, wherein the plurality of execution metrics is based on the telemetry data, represent overall performance of the executing of the batch job on the plurality of virtual compute instances, and comprise a job duration metric and a queue depth metric indicating an average number of jobs in the job queue over time;
• outputting a second scheduling policy by the batch-processing service utilizing one or more models and the plurality of execution metrics, wherein the one or more models have been trained using training data for batch jobs executing on the plurality of virtual compute instances with an average run time less than a permissible threshold;
• automatically executing at least one subsequent batch job on the plurality of virtual compute instances on behalf of the user account using the second scheduling policy, wherein the second scheduling policy is more optimized than the first scheduling policy to schedule subsequent batch jobs on behalf of the user account;
• and
• providing the user account with access to the second scheduling policy and the visualization of the plurality of execution metrics.

A method comprising:

• receiving, at a batch-processing service, a request to execute a batch job on behalf of a user account registered with the batch-processing service;
• causing execution of the batch job on a plurality of virtual compute instances associated with the batch-processing service according to a first scheduling policy of the user account;
• receiving telemetry data associated with the execution of the batch job on the plurality of virtual compute instances, wherein the telemetry data is based on amounts of computing resources utilized by the batch job during the execution;
• generating a visualization that includes a graphical representation of a plurality of execution metrics over time for the batch job, wherein the plurality of execution metrics is based on the telemetry data, represent overall performance of the executing of the batch job on the plurality of virtual compute instances, and comprise a job duration metric and a queue depth metric indicating an average number of jobs in the job queue over time;
• outputting a second scheduling policy by the batch-processing service utilizing one or more models and the plurality of execution metrics, wherein the one or more models have been trained using training data for batch jobs executing on the plurality of virtual compute instances with an average run time less than a permissible threshold;
• automatically executing at least one subsequent batch job on the plurality of virtual compute instances on behalf of the user account using the second scheduling policy, wherein the second scheduling policy is more optimized than the first scheduling policy to schedule a plurality of subsequent batch jobs on behalf of the user account;
• and
• providing the user account with access to the second scheduling policy and the visualization of the plurality of execution metrics.