Method and system for the dynamic scheduling of a stream of computing jobs based on priority and trigger threshold

A non-transitory computer readable medium storing instructions to execute jobs, the instructions comprising functionality to:

• obtain a first plurality of jobs having a first plurality of priorities;
• generate, based on the first plurality of priorities, a first schedule for executing the first plurality of jobs;
• identify a lowest priority in the first plurality of priorities;
• set a first trigger threshold to a value exceeding the lowest priority;
• initiate execution of the first plurality of jobs according to the first schedule;
• obtain a second plurality of jobs having a second plurality of priorities during execution of at least one of the first plurality of scheduled jobs, wherein the second plurality of priorities comprises a priority satisfying the first trigger threshold;
• generate, based on the second plurality of priorities and in response to the priority satisfying the first trigger threshold, a second schedule for executing the second plurality of jobs and a job of the first plurality of jobs;
• set a second trigger threshold based on at least one of the second plurality of priorities;
• and
• initiate execution of the second plurality of jobs and the job of the first plurality of jobs according to the second schedule.

A system for executing jobs comprising:

• a hardware processor;
• a priority generator configured to assign a first plurality of priorities to a first plurality of jobs and a second plurality of priorities to a second plurality of jobs;
• and
• a scheduler executing on the hardware processor and configured to:
• generate, based on the first plurality of priorities, a first schedule for executing the first plurality of jobs;
• identify a lowest priority in the first plurality of priorities;
• set, before the second plurality of priorities are assigned to the second plurality of jobs, a first trigger threshold to a value exceeding the lowest priority, wherein the second plurality of priorities includes a priority satisfying the first trigger threshold;
• and
• generate, based on the second plurality of priorities and in response to the priority satisfying the first trigger threshold, a second schedule for executing the second plurality of priorities and a job of the first plurality of jobs;
• and
• a plurality of resources configured to execute the first plurality of jobs according to the first schedule and to execute the second plurality of jobs and the job of the first plurality of jobs according to the second schedule.