System and method for using sparsity to accelerate deep learning networks

A system for using sparsity to accelerate deep learning networks, the system comprising one or more processing units in communication with data memory for executing processing elements (PEs), each processing element receiving a plurality of input streams and comprising:

• staging buffers for receiving and storing input tensors respectively from the plurality of input streams for subsequent multiply-accumulate (MAC) operations;
• a scheduler to receive a bit vector from each staging buffer to identify which values in a respective one of the input tensors are non-zero and to determine, based on the bit vectors, locations of non-zero values;
• wherein, for each MAC operation, the scheduler selects, for each corresponding lane of the input tensors, one of the values of the input tensor to form a non-zero value pair and be communicated to a MAC a unit, the selection of the non-zero values comprising one of:
• the current value in the lane;
• the next value in the same lane where such value is non-zero;
• a value from a step ahead in time where such value is non-zero;
• and
• a value from a neighboring lane where such value is non-zero;
• and
• performing and outputting the values of the MAC operations by the MAC unit using the selected values.