Auto recognition of big data computation engine for optimized query runs on cloud platforms

A computing system, comprising:

• one or more processors;
• and
• a computer readable memory comprising instructions that, when executed by the one or more processors, cause the computing system to perform operations to:
• receive, by the computing system, a request for a big data query comprising one or more characteristics and one or more user parameters;
• access, by the computing system, a set of rules, the set of rules being based at least in part on respective properties of one or more big data query engines, and the set of rules correlating at least one of the one or more characteristics associated with the big data query and the one or more user parameters with the respective properties of the one or more big data query engines;
• determine, by the computing system, a candidate list of big data query engines comprising a subset of the one or more big data query engines, the candidate list determined based at least in part on the set of rules;
• generate, by the computing system and using a machine learning model respective probability scores for each big data query engine of candidate list of big data query engines, the respective probability scores representing a likelihood of the big data query being successfully completed by each big data query engine of the subset;
• select, by the computing system, a particular big data query engine of the one or more big data query engines of the candidate list, based at least in part on the respective probability score of the particular big data query engine;
• execute, by the computing system, the big data query using the particular big data query engine;
• identify, by the computing system, a trigger indicating a performance issue with the particular big data query engine;
• and
• switch, by the computing system, the execution the big data query to a second big data query engine of the candidate list of big data query engines.

A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform operations comprising:

• receiving, by a computing system, a request for a big data query comprising one or more characteristics and one or more user parameters;
• accessing, by the computing system, a set of rules, the set of rules being based at least in part on respective properties of one or more big data query engines, and the set of rules correlating at least one of the one or more characteristics associated with the big data query and the one or more user parameters with the respective properties of the one or more big data query engines;
• determining, by the computing system, a candidate list of big data query engines comprising a subset of the one or more big data query engines, the candidate list determined based at least in part on the set of rules;
• generating, by the computing system and using a machine learning model, respective probability scores for each big data query engine of candidate list of big data query engines, the respective probability scores representing a likelihood of the big data query being successfully completed by each big data query engine of the subset;
• selecting, by the computing system, a particular big data query engine of the one or more big data query engines of the candidate list, based at least in part on the respective probability score of the particular big data query engine;
• and
• executing, by the computing system, the big data query using the particular big data query engine;
• identifying, by the computing system, a trigger indicating a performance issue with the particular big data query engine;
• and
• switching, by the computing system, the execution the big data query to a second big data query engine of the candidate list of big data query engines.