Cost-optimal cluster configuration analytics package

A system comprising:

• one or more processors;
• and
• at least one computer-readable storage medium having stored therein instructions which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
• receiving job input including one or more applications and a sample of a production load of a job to be outsourced to a remote cluster computing system;
• generating an application recommendation vector for the job independent of cluster configuration in the remote cluster computing system using the job input;
• instantiating different clusters of nodes to form different cluster configurations in the remote cluster computing system and the different cluster configurations differ by varying one or a combination of hardware parameters of one or more nodes in the cluster of nodes to form the different cluster configurations, a number of nodes of the one or more nodes in the cluster of nodes to form the different cluster configurations, and resource allocation of the one or more nodes in the cluster of nodes to form the different cluster configurations;
• forecasting the job in the remote cluster computing system by identifying expected completion times of the production load across varying cluster configurations using one or more multi-linear regression models segmented into parts by different load regions, wherein the one or more multi-linear regression models are trained by running at least a portion of the sample of the production load on the clusters of nodes with different cluster configurations in the remote cluster computing system using the one or more applications based on the application recommendation vector;
• and
• identifying an optimal cluster configuration of the varying cluster configurations for the job in the remote cluster computing system based on the identified expected completion times of the production load across the varying cluster configurations.

A non-transitory computer-readable storage medium having stored therein instructions which, when executed by a processor, cause the processor to perform operations comprising:

• receiving job input including one or more applications and a sample of a production load of a job to be outsourced to a remote cluster computing system;
• instantiating different clusters of nodes to form different cluster configurations in the remote cluster computing system;
• forecasting the job in the remote cluster computing system by identifying expected completion times of the production load across varying cluster configurations using one or more multi-linear regression models segmented into parts by different load regions, wherein the one or more multi-linear regression models are trained by running at least a portion of the sample of the production load on the different clusters of nodes with the different cluster configurations in the remote cluster computing system using the one or more applications;
• and
• identifying an optimal cluster configuration of the varying cluster configurations for the job in the remote cluster computing system based on the identified expected completion times of the production load across the varying cluster configurations.