Adaptive scheduling

A method for operating a broker that assigns tasks to available processing resources, the method comprising:

• determining, in response to an indication of availability received from a computing resource, whether the computing resource has an associated discriminator;
• if an associated discriminator is found, using the discriminator to eliminate from consideration task(s) that do not qualify for assignment to the computing resource;
• and,
• if any task(s) that qualify for assignment remain, employing a task assignment strategy to assign one of the qualified task(s) to the computing resource.

In a distributed computing system, a method for minimizing unnecessary data communication, comprising:

• allowing available processing resources to communicate their availability to a task broker;
• and,
• using the task broker to assign tasks to available computing resources;
• characterized in that at least some of the available processing resources, when communicating their availability to the broker, also communicate information concerning locally-cached data, which information is used by the broker to preferentially assign task(s) to processing resource(s) on which data needed by the task(s) is locally cached.

In a distributed computing system, a method for assigning tasks to processing resources, comprising:

• (a) identifying a processing resource available to accept a task;
• (b) identifying task(s) that may be assigned to the processing resource;
• (c) determining which of the identified task(s) to assign to the processing resource based on consideration of at least two of:
• the operating system of the processing resource vs. the operating system requirements of the task;
• the available memory of the processing resource vs. the memory requirements of the task;
• the available disk space of the processing resource vs. the disk space requirements of the task;
• the security features of the processing resource vs. the security requirements of the task;
• the speed of the processing resource vs. the processing demands of the task;
• and,
• the availability of locally-cached data at the processing resource vs. the data set requirements of the task;
• and,
• (d) repeating (a)-(c), as needed, to complete assignment of all tasks to one or more processing resources.

For use in a distributed computing system, a broker that manages the execution of jobs, each job having one or more associated tasks that can be executed in any order, the broker comprising:

• a tracking module that, for each active job, tracks associated:
• (i) waiting tasks that have not yet been assigned to a processing resource;
• (ii) pending tasks that have been assigned to at least one processing resource, but have not yet been completed by any processing resource;
• and (iii) completed tasks that have been completed by at least one processing resource;
• and,
• an adaptive scheduling module that maintains a priority for each active job and, in response to an indication of availability from a processing resource, selects a job and assigns at least one task associated with the selected job to the processing resource.

A broker, configured to assign tasks to available processing resources in a network-based distributed computing system, the broker comprising:

• a resource evaluator which, in response to an indication of availability received from a processing resource, determines at least two of the following attributes of the available processing resource:
• operating system of the processing resource;
• available memory of the processing resource;
• available disk space of the processing resource;
• security features of the processing resource;
• speed of the processing resource;
• availability of locally-cached data at the processing resource;
• typical frequency of local user activity at the processing resource;
• and,
• time of most recent local user activity at the processing resource;
• and,
• a resource assignor which, in response to an indication of availability received from a processing resource, determines which task to assign to the available processing resource based, at least in part, on at least two of the following attributes of the task:
• preferred or required operating system for executing the task;
• preferred or required memory for executing the task;
• preferred or required disk space for executing the task;
• preferred or required security features for executing the task;
• preferred or required processor speed for executing the task;
• preferred or required availability of locally-cached data at the processing resource;
• and,
• preferred or required duration of uninterrupted control of the processing resource while executing the task.

A discriminator, configured for use by a broker in determining assignments of tasks to available resources, the discriminator embodied on a computer-readable medium, and comprising a numeric function, associated with either a task or a processing resource, which discriminator may be called to return a numeric value computed by combining information concerning at least two of the following:

• whether the operating system of a particular processing resource is compatible with the operating system requirements of a particular task;
• whether the memory available at a particular processing resource is sufficient to execute a particular task;
• whether the disk space available at a particular processing resource is sufficient to execute a particular task;
• whether the security infrastructure of a particular processing resource is sufficient to meet the security requirements of a particular task;
• whether the speed of a particular processing resource is sufficient to meet the processing demands of a particular task;
• whether particular data set or state information needed to execute a particular task is already locally cached at a particular processing resource;
• and,
• whether the level of local processing demand at a particular processing resource is sufficiently low to accommodate a particular task.

A discriminator, configured for use by a broker in determining assignments of tasks to available resources, the discriminator embodied on a computer-readable medium, and comprising a binary function, associated with either a task or a processing resource, which discriminator may be called to return a binary value computed, at least in part, as a product of at least two binary values, each value indicative of one of the following conditions:

• whether the operating system of a particular processing resource is compatible with the operating system requirements of a particular task;
• whether the memory available at a particular processing resource is sufficient to execute a particular task;
• whether the disk space available at a particular processing resource is sufficient to execute a particular task;
• whether the security infrastructure of a particular processing resource is sufficient to meet the security requirements of a particular task;
• whether the speed of a particular processing resource is sufficient to meet the processing demands of a particular task;
• whether particular data set or state information needed to execute a particular task is already locally cached at a particular processing resource;
• and,
• whether the level of local processing demand at a particular processing resource is sufficiently low to accommodate a particular task.