Multi substrate public cloud big data jobs as a service

An apparatus for data processing at an application server, comprising:

• a processor,
• memory coupled with the processor;
• and
• instructions stored in the memory and executable by the processor to cause the apparatus to:
• receive a configuration indicating a plurality of parameters for performing a data processing job and a plurality of performance targets for performing the data processing job;
• identify available compute resources from a plurality of public cloud infrastructures, wherein each public cloud infrastructure of the plurality of public cloud infrastructures supports one or more computing applications, one or more job schedulers, and one or more utilization rates associated with the one or more job schedulers;
• select one or more compute clusters from one or more of the plurality of public cloud infrastructures based at least in part on a matching process between the parameters for performing the data processing job and a combination of the one or more computing applications, the one or more job schedulers, and the one or more utilization rates associated with the one or more job schedulers, wherein the matching process comprises one or more rules configured at the application server, the one or more rules defining one or more weights for each of the plurality of parameters for performing the data processing job, the one or more weights associated with the plurality of performance targets and based at least in part on one or more locations, at one or more public cloud infrastructures of the plurality of public cloud infrastructures, of data to be processed by the data processing job, and wherein the one or more compute clusters are selected based at least in part on the weights and a prioritization of the plurality of performance targets, the prioritization based at least in part on the one or more weights;
• and
• initiate the one or more compute clusters for processing the data processing job based at least in part on the selecting.

A non-transitory computer-readable medium storing code for data processing at an application server, the code comprising instructions executable by a processor to:

• receive a configuration indicating a plurality of parameters for performing a data processing job and a plurality of performance targets for performing the data processing job;
• identify available compute resources from a plurality of public cloud infrastructures, wherein each public cloud infrastructure of the plurality of public cloud infrastructures supports one or more computing applications, one or more job schedulers, and one or more utilization rates associated with the one or more job schedulers;
• select one or more compute clusters from one or more of the plurality of public cloud infrastructures based at least in part on a matching process between the parameters for performing the data processing job and a combination of the one or more computing applications, the one or more job schedulers, and the one or more utilization rates associated with the one or more job schedulers, wherein the matching process comprises one or more rules configured at the application server, the one or more rules defining one or more weights for each of the plurality of parameters for performing the data processing job, the one or more weights associated with the plurality of performance targets and based at least in part on one or more locations, at one or more public cloud infrastructures of the plurality of public cloud infrastructures, of data to be processed by the data processing job, and wherein the one or more compute clusters are selected based at least in part on the weights and a prioritization of the plurality of performance targets, the prioritization based at least in part on the one or more weights;
• and
• initiate the one or more compute clusters for processing the data processing job based at least in part on the selecting.