Method, apparatus, and computer program product for scheduling work in a stream-oriented computer system with configurable networks

A non-transitory computer program product for scheduling stream-based applications in a distributed computer system with configurable networks, the computer program product comprising:

• a storage medium readable by a computer processor and storing instructions for execution by the computer processor for implementing a method, the method comprising:
• choosing, at a highest temporal level, jobs having a priority assigned at the highest temporal level and that will run, an optimal template alternative for the jobs that will run, network topology on which the jobs will run, and candidate processing nodes for processing elements of the optimal template alternative for each job that will run;
• determining an importance of work at the highest temporal level of each job that will run based on the priority of each job assigned at the highest temporal level, determining an overall importance of the work performed by the system based on the importance of work at the highest temporal level of each job that will run, and maximizing the overall importance of work performed by the system based on the choosing, at the highest temporal level, of the jobs that will run, the optimal template alternative, the network topology on which the chosen jobs will run, and the candidate processing nodes;
• making, at a medium temporal level, fractional allocations and re-allocations of the processing elements to the candidate processing nodes chosen at the highest temporal level in the system to react to changing of the importance of work determined at the highest temporal level;
• and
• revising, at a lowest temporal level, the fractional allocations and re-allocations performed at the medium temporal level on a continual basis to react to burstiness of the work, and to differences between projected and real progress of the work.

A method for scheduling stream-based applications in a distributed computer system, comprising:

• providing a scheduler configured to schedule work using three temporal levels;
• configuring the network and scheduling the jobs that will run, in a highest temporal level, in accordance with a plurality of operation constraints determined at the highest temporal level, the plurality of operation constraints including a priority assigned to each job that will run;
• determining an importance of work at the highest temporal level of each job that will run based on the priority of each job determined at the highest temporal level, determining an overall importance of the work performed by the system based on the importance of work at the highest temporal level of each job that will run to optimize an overall importance of work of the distributed computer system;
• fractionally allocating, at a medium temporal level, processing elements to processing nodes chosen at the highest temporal level in the system to react to changing importance of the work determined at the highest temporal level;
• and
• revising, at a lowest temporal level, the fractional allocations performed at the medium temporal level on a continual basis to react to burstiness of the work, and to differences between projected and real progress of the work.

An apparatus for scheduling stream-based applications in a distributed computer system with configurable networks, comprising:

• a scheduler computer configured to schedule work using three temporal levels including:
• a macro method configured to determine network topology on which jobs will run and schedule the jobs that will run, in a highest temporal level, in accordance with a plurality of operation constraints determined at the highest temporal level, the plurality of operation constraints including a priority assigned to each job that will run;
• determining an importance of work at the highest temporal level of each job that will run based on the priority of each job determined at the highest temporal level, determining an overall importance of the work performed by the system based on the importance of work at the highest temporal level of each job that will run to optimize an overall importance of work of the distributed computer system;
• a micro method configured to fractionally allocate, at a medium temporal level, processing elements to processing nodes chosen at the highest temporal level in the system to react to changing importance of the work determined at the highest temporal level;
• and
• a nano method configured to revise, at a lowest temporal level, the fractional allocations performed at the medium temporal level on a continual basis to react to burstiness of the work, and to differences between projected and real progress.