Method and system for generating a target pattern-based optimal scheduling policy

A computing device for generating a target pattern-based optimal scheduling policy for a set of jobs of an application, the computing device comprising:

• a processor;
• a memory;
• and
• a communication interface coupled to each of the processor and the memory, wherein the processor is configured to:
• receive, via the communication interface from a repository, a historical telemetry data associated with the set of jobs,
• generate, via a workload pattern builder, a workload pattern of the set of jobs for each calendar day based on the historical telemetry data associated with the set of jobs,
• receive, at a job metadata store, a metadata store information of the set of jobs,
• execute, via a mapper engine using a reinforcement learning technique, a set of actions based on the workload pattern of the set of jobs and the metadata store information of the set of jobs,
• generate, via the mapper engine, an intermediate policy for the set of jobs based on the execution of the set of actions,
• execute, via a scheduling engine, the set of jobs on a set of respective run machines based on the workload pattern of the set of jobs and the intermediate policy,
• determine, via an event capture engine, a feedback data based on the execution of the set of jobs on the set of respective run machines, and
• generate, via an optimizer engine using the reinforcement learning technique, the target pattern-based optimal scheduling policy for the set of jobs based on at least one among the feedback data, and the set of actions.

A non-transitory computer readable storage medium storing instructions for generating a target pattern-based optimal scheduling policy for a set of jobs of an application, the storage medium comprising executable code which, when executed by a processor, causes the processor to:

• receive, via a communication interface from a repository, a historical telemetry data associated with the set of jobs;
• generate, via a workload pattern builder, a workload pattern of the set of jobs for each calendar day based on the historical telemetry data associated with the set of jobs;
• receive, at a job metadata store, a metadata store information of the set of jobs;
• execute, via a mapper engine using a reinforcement learning technique, a set of actions based on the workload pattern of the set of jobs and the metadata store information of the set of jobs;
• generate, via the mapper engine, an intermediate policy for the set of jobs based on the execution of the set of actions;
• execute, via a scheduling engine, the set of jobs on a set of respective run machines based on the workload pattern of the set of jobs and the intermediate policy;
• determine, via an event capture engine, a feedback data based on the execution of the set of jobs on the set of respective run machines;
• and
• generate, via an optimizer engine using the reinforcement learning technique, the target pattern-based optimal scheduling policy for the set of jobs based on at least one among the feedback data, and the set of actions.