System and method for input data load adaptive parallel processing

A method, implemented on hardware logic and/or on software logic executing on processors, for executing a plurality of application programs, the method comprising:

• on each of a plurality of data input buffers, queuing respective data for a corresponding program of the plurality of application programs;
• periodically allocating an array of units of computing capacity among the plurality of application programs, wherein allocating comprises, for each period of a plurality of allocation periods,
• identifying expressions of demand for the array of units of computing capacity by each program of the plurality of application programs, wherein
• the respective expression of demand for a given program of the plurality of application programs is based at least in part on quantifying the respective data queued for the given program in one or more input buffers of the plurality of data input buffers corresponding to the given program as a count of processing data units, and
• allocating the array of units of computing capacity based at least in part on the expressions of demand of the plurality of application programs;
• and
• for each period of a plurality of allocation periods, responsive to the allocating,
• assigning, for each program of the plurality of application programs, one or more instances of the respective program for execution on a respective one or more units of the array of units of computing capacity, wherein
• the one or more instances are assigned according to i) a respective number of the units of the array of units of computing capacity allocated to the respective program, and ii) execution priorities among instances of the respective program, and
• according to the assigning, establishing direct data access from each buffer of at least a subset of the plurality of data input buffers to the cores respective unit of the array of units of computing capacity that is assigned a given corresponding instance of the respective program for which the respective data buffer is queuing input data, wherein
• establishing direct data access comprises enabling the given corresponding instance to access and process the respective input data at the respective data buffer.

A system, executing on at least one of hardware logic and software logic executing on a plurality of processors, for hosting a plurality of application programs, the system comprising:

• a plurality of data buffers, each buffer of the plurality of data buffers queuing input data for a respective program of the plurality of application programs, wherein
• the respective program is configured to access and process the input data during execution of the respective program;
• a plurality of processing units;
• a first subsystem configured to allocate the plurality of processing units among the plurality of application programs, wherein allocating comprises
• allocating the plurality of processing units based at least in part on a respective volume of input data at each buffer of the plurality of data buffers,
• and
• allocating, for at least some time periods, more than one processing unit of the plurality of processing units to at least one of the plurality of application programs;
• a second subsystem configured to, for each program of the plurality of application programs, select one or more instances of a plurality of instances of the respective program, wherein
• selecting is based at least in part on respective volumes of input data available for each instance of the plurality of instances of the respective program, and
• a different instance of the respective program is selected for each processing unit allocated to the respective program;
• a third subsystem configured to assign, for each program of the plurality of application programs, each processing unit allocated to the respective program to a different instance of the one or more instances of the respective program in accordance with the selecting;
• and
• a fourth subsystem configured, according to the assigning, to establish direct data access from each buffer of at least a subset of the plurality of data buffers to the respective processing unit of the plurality of processing units that is assigned a given corresponding instance of the respective program for which the respective data buffer is queuing input data, wherein
• establishing direct data access comprises enabling the given corresponding instance to access and process the respective input data at the respective data buffer;
• wherein the plurality of processing units is periodically allocated by the first subsystem, selected by the second subsystem, and assigned by the third subsystem based at least in part on changes in respective volumes of input data associated with each program of the plurality of application programs.