Load balancing on heterogeneous processing clusters implementing parallel execution

A computer readable storage medium comprising a computer readable program, wherein the computer readable program when executed on a computer causes the computer to perform a method for managing data loads on a cluster of processors that implement an iterative procedure through parallel processing of data for the procedure, the method comprising:

• for at least one iteration of the procedure, monitoring completion times of a plurality of different processing phases that are undergone by each of the processors in a given iteration of the at least one iteration;
• determining whether a load imbalance factor threshold is exceeded in the given iteration based on the completion times for the given iteration;
• and
• repartitioning the data by reassigning the data to the processors based on predicted dependencies between assigned data units of the data and completion times of a plurality of the processers for at least two of the phases.

A system for managing data loads comprising:

• a cluster of processors configured to implement an iterative procedure through parallel processing of data for the procedure;
• a data repartitioner module configured to partition and assign the data to the processors for the parallel processing;
• and
• a balancer module configured to, for at least one iteration of the procedure, monitor completion times of a plurality of different processing phases that are undergone by each of the processors in a given iteration of the at least one iteration, to determine whether a load imbalance factor threshold is exceeded in the given iteration based on the completion times for the given iteration, and to direct the data repartitioner module to repartition the data by reassigning the data to the processors based on predicted dependencies between assigned data units of the data and completion times of a plurality of the processers for at least two of the phases.