Neural hardware accelerator for parallel and distributed tensor computations

A method comprising:

• retrieving locally or receiving a portion of an input data tensor at a neural core of a plurality of neural cores, the neural core comprising a memory, the plurality of neural cores being arranged in a two-dimensional array, and the plurality of neural cores including direct connections and relay connections with other of the plurality of neural cores;
• reading from the memory at least a portion of a weight tensor, the weight tensor comprising a plurality of filters, the weight tensor being distributed among the plurality or neural cores according to physical dimensions of the two-dimensional array and based on a locality of the weight tensor and the direct connections of each of the plurality of neural cores, and the dimensions, shape, and partitioning of the weight tensor being configurable,
• wherein the distribution of the weight tensor is according to a ratio of a number of the direct connections to a number of the relay connections;
• wherein the ratio is maximized for the direct connections to the relay connections when transmitting the weight tensor between the plurality of neural cores;
• applying the portion of the weight tensor to the portion of the input data tensor to obtain a result;
• storing locally or sending the result via a network to at least one other neural core.

A system comprising:

• a plurality of neural cores, each of the plurality of neural cores comprising a memory,
• wherein the plurality of neural cores is arranged in a two-dimensional array, and wherein each of the plurality of neural cores includes direct connections and relay connections with other of the plurality of neural cores;
• a network interconnecting the plurality of neural cores;
• a computing node comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor of the computing node to cause the processor to perform a method comprising:
• encoding a plurality of filters in a weight tensor;
• and
• providing at least a portion of the weight tensor to each of the plurality of neural cores,
• wherein the weight tensor is distributed among the plurality of neural cores according to physical dimensions of the two-dimensional array and based on a locality of the weight tensor and the direct connections of each of the plurality of neural cores, and the dimensions, shape, and partitioning of the weight tensor are configurable, wherein the
• distribution of the weight tensor is according to a ratio of a number of the direct connections to a number of the relay connections,
• wherein the ratio is maximized for the direct connections to the relay connections when transmitting the weight tensor between the plurality of neural cores, and
• wherein each of the plurality of neural cores is adapted to:
• store in its memory at least a portion of the weight tensor;
• retrieve locally or receive a portion of an input data tensor;
• apply the portion of the weight tensor to the portion of the input data tensor to obtain a result;
• and
• store locally or send the result via a network to at least one other neural core.