Job scheduling using expected server performance information

A cluster system comprising:

• a plurality of servers, each of at least some of which including an expected server performance monitoring component configured to maintain expected server performance information comprising expected performance parameters of tasks having varying characteristics if assigned to the server;
• a cluster management component configured to maintain cluster-level information;
• a plurality of job scheduler components, each of at least some of the plurality of job scheduler components comprising:
• an interface for receiving expected server performance information from the plurality of servers, the expected server performance information for a given server comprising expected performance parameters for performance of tasks of each of a variety of task categories if assigned to the given server, the interface also for receiving cluster-level information associated with a cluster of servers that include the plurality of servers;
• and
• a job management component that is configured to manage a job assignment associated with a computational job by identifying a plurality of tasks associated with the computational job, identifying task dependencies in performance of the plurality of tasks, determining which of the plurality of tasks are ready for execution based on the dependencies, and for each of at least some tasks that are ready for execution, performing the following:
• an act of identifying a particular task category of the corresponding task;
• an act of using expected server performance information for the particular task category received through the interface, and using the cluster-level information received through the interface to select a server from amongst the plurality of servers to perform the corresponding task;
• and
• an act of submitting a request to perform the task to the selected server.

A method for a job management component to manage a job assignment associated with a computational job, the method comprising:

• an act of identifying a plurality of tasks associated with the computational job;
• an act of identifying task dependencies in the plurality of tasks;
• an act of determining which of the plurality of tasks are ready for execution based on the task dependencies,
• for each of at least some tasks that are ready for execution, performing the following:
• an act of identifying a particular task category of the corresponding task;
• an act of receiving expected server performance information for the particular task category received through an interface;
• an act of receiving cluster-level information through an interface;
• an act of using the received expected server performance information for the particular task category and the received cluster-level information to select a server from amongst the plurality of servers to perform the corresponding task;
• and
• an act of submitting a request to perform the corresponding task to the selected server.