Neural network accelerator

A computing element array, comprising:

• an array of computing elements connected by connections, wherein each computing element comprises a control circuit, a storage circuit, and an operation circuit, and each computing element is operable to store a self-tag that identifies an operation performed by the operation circuit, and
• wherein for each computing element in the array of computing elements,
• the storage circuit is operable to receive a data packet from one of the connections and store the data packet,
• the data packet comprises a data value and a target-tag specifying an operation to be performed on the data value,
• the control circuit is operable to compare the target-tag to the self-tag, enable the operation circuit to process the data value if the self-tag matches the target-tag, modify the data packet to comprise the processed data value, and enable the output of the modified data packet on one of the connections, and
• the operation circuit is operable to process the data value to form a processed data value,
• wherein the array is a three-dimensional grid, a three-dimensional array a torus, or a ring with one or more uni-directional connections.

A computing element array, comprising:

• an array of computing elements connected by connections, wherein each computing element comprises a control circuit, a storage circuit, and an operation circuit, and
• wherein for each computing element in the array of computing elements,
• the storage circuit is operable to receive a data packet from one of the connections and store the data packet,
• the data packet comprises a data value and a target-tag specifying routing values, wherein one of the routing values is a no-route value indicating that the operation circuit operates on the data value,
• the control circuit is operable to identify the no-route value, enable the operation circuit to process the data value if the no-route value is identified, modify the data packet to comprise the processed data value, and enable the output of the modified data packet on one of the connections, and
• the operation circuit is operable to process the data value to form a processed data value,
• wherein the array is a three-dimensional grid, a three-dimensional array, a torus, or a ring with uni-directional connections.

A computing element array system, comprising:

• an array of computing elements connected by connections, wherein each computing element comprises a control circuit, a storage circuit, and an operation circuit, and
• wherein the connections each connect two computing elements, and
• wherein for each computing element in the array of computing elements,
• the storage circuit is operable to input a data packet comprising a data value and a target-tag from one of the connections and to store the data packet,
• the operation circuit is operable to perform an operation on the data value to form a processed data value,
• the target-tag specifies a computing element to perform the operation on the data value, and
• the control circuit is operable to identify the computing element with the computing element specified by the target-tag, enable the operation circuit to process the data value if the computing element identified by the target-tag is the computing element, modify the data packet to comprise the processed data value, and enable the output of the modified data packet on one of the connections,
• wherein the computing elements are not all identical and have different operation circuits.