Systolic neural network engine capable of forward propagation

A method for performing computations of a neural network comprising at least a first layer and a second layer, the method comprising:

• assigning a first data processing unit (DPU) to perform computations of a first node of the first layer of the neural network;
• assigning a second DPU to perform computations of a second node the second layer of the neural network;
• performing computations of the first node of the first layer using the first DPU to generate a first activation output;
• transmitting the first activation output to a first output systolic element of the first DPU;
• systolically pulsing the first activation output from the first output systolic element to a first input systolic element of the second DPU during a first systolic pulse;
• and
• performing computations of the second node of the second layer by using the second DPU to process at least the first activation output,
• wherein the method is performed by at least one processor.

A non-transitory computer-readable medium storing computer-executable instructions that, when executed by a processor, cause the processor to perform operations comprising:

• performing, using a first processing unit, computations of a first node of a neural network to generate a first activation output, the first node included in a first layer of the neural network;
• systolically pulsing the first activation output from the first processing unit to a second processing unit assigned to perform computations of a second node of the neural network, the second node included in a second layer of the neural network;
• and
• performing computations of the second node by using the second processing unit to process at least the first activation output to generate a second activation output.