Application load adaptive multi-stage parallel data processing architecture

A system including processors and implemented on hardware logic and/or software logic executing on the processors, comprising a plurality of subsystems for managing access to a pool of processing resources by application programs, the system comprising:

• a first subsystem configured to allocate a plurality of processing cores of the pool of processing resources to a plurality of application programs based at least in part on
• respective expressions of demand for processing resources of the plurality of processing cores by the plurality of application programs and respective entitlements for the processing resources by a subset of the plurality of application programs, wherein
• the respective expressions of demand relate in part to readiness to execute of one or more instances of a respective application program, and
• the plurality of processing cores comprises at least two types of processing cores and/or cores of a reconfigurable type;
• a second subsystem configured to, for each program of the plurality of application programs, select a set of highest priority instances of one or more instances of the respective application program, wherein each of the set of instances is selected based at least in part on a) the number of cores allocated to the respective program and b) a relative readiness for execution among the one or more instances of the respective application program;
• and
• a third subsystem configured to assign the respective set of highest priority instances of each of the plurality of application programs for execution on a respective one or more processing cores of the plurality of processing cores, wherein assigning is based at least in part upon a type or configuration of processing core demanded by the respective application program instance;
• wherein allocating the plurality of processing cores by the first subsystem comprises adjusting allocations of the plurality of processing cores over time based at least in part on changes in the respective expressions of demand for the processing resources of the plurality of processing cores,
• wherein assigning further comprises prioritizing placement of the set of highest priority instances to the particular processing core type or configuration demanded, and
• wherein the system periodically repeats the selecting and the assigning at least in response to the adjusting allocations.

A method for managing resource access on a parallel processing system on behalf of application programs, the method comprising:

• identifying, by a control system comprising at least one of hardware logic and software logic, a respective expression of demand for computing capacity of the parallel processing system associated with each of one or more programs of a plurality of application programs;
• identifying, by the control system, a respective entitlement for computing capacity associated with each of one or more programs of the plurality of application programs;
• allocating, by the control system, a plurality of units of the computing capacity of the parallel processing system to at least a subset of the plurality of application programs, wherein
• the plurality of units comprises units of at least two different types and/or units of a reconfigurable type, and
• the allocating is based at least in part on i) the expressions for demand of each of the subset of the plurality of application programs, and ii) the entitlements of the one or more programs of the subset of application programs;
• for each program of the subset of application programs,
• assessing, by the control system, a relative readiness for execution among one or more instances of the respective application program, and
• selecting, by the control system, a set of highest priority instances of the one or more instances of the respective application program for execution on the parallel processing system according to a) a number of units of the plurality of units of computing capacity allocated to the respective application program, and b) the relative readiness for execution among the one or more instances of the respective application program;
• assigning, by the control system, the set of the highest priority instances of each of the at least the subset of the plurality of application programs for execution on a respective one or more units of the plurality of units of computing capacity, wherein
• assigning comprises, for each instance of the set of highest priority instances expressing a demand for a particular type or configuration of unit, prioritizing placement of the respective instance to the particular type or configuration of unit demanded;
• and
• adjusting, by the control system, allocations of the plurality of units of computing capacity over time based at least in part on changes in the expressions of demand or the entitlements of the plurality of application programs, wherein adjusting comprises periodically repeating the identifying of the respective expression of demand, the identifying the respective entitlement, the allocating, the assessing, the selecting, and the assigning.