Scalable acceleration of reentrant compute operations

An IC, comprising:

• a plurality of PEs comprising circuitry, each of the plurality of PEs comprising a local memory for storing a respective data chunk of a data set, wherein the plurality of PEs is configured to perform an algorithm on the data chunks that includes an operation which requires each of the data chunks;
• a controller comprising circuitry, the controller configured to:
• receive at least one token from the plurality of PEs indicating they have finished a first iteration of processing the data chunks in parallel;
• receive partial results generated by the plurality of PEs based on performing the first iteration;
• determine intermediate results by performing the operation based on the partial results;
• and
• distribute the intermediate results to the plurality of PEs;
• wherein the plurality of PEs is configured to perform a second iteration to process the data chunks in parallel based on the intermediate results.

A method, comprising:

• processing, during a first iteration and in parallel, data chunks of a data set at respective ones of a plurality of PEs, wherein the first iteration is part of an operation that has a data dependency among the data chunks;
• transmitting, from at least one PE of the plurality of PEs, a token to a controller indicating the plurality of PEs have finished the first iteration;
• receiving, at the controller, partial results generated by the plurality of PEs based on performing the first iteration;
• determining intermediate results from the partial results to resolve the data dependency;
• distributing the intermediate results to the plurality of PEs;
• and
• processing, during a second iteration and in parallel, the data chunks and the intermediate results at the plurality of PEs.