Neural network hardware accelerator system with zero-skipping and hierarchical structured pruning methods

A method for pruning an artificial neural network (ANN), comprising:

• identifying a target ratio of zero weights for a layer of the ANN;
• identifying a limited zero sequence (LZS) condition for the layer, wherein the LZS condition comprises a limitation on a location or sequence length of zeroes in the layer;
• pruning the layer based at least in part on the target ratio of zero weights by setting one or more weights of the layer to zero;
• determining whether each sequence of zero weights in the layer violates the LZS condition after the pruning;
• and
• resetting at least one weight within each sequence of zero weights that violates the LZS condition to a non-zero value.

A computing system for processing an artificial neural network (ANN), comprising:

• one or more memory units configured to store weights of the ANN and values of an input tensor;
• and
• a plurality of accelerator sub-systems, wherein each accelerator subsystem further comprises:
• a multiplexing buffer configured to store weights of the ANN;
• a look-ahead buffer configured to store additional weights of the ANN;
• a zero-skipping circuit configured to identify a next non-zero weight of the ANN;
• and
• a plurality of multiply-accumulate (MAC) units configured to simultaneously process the next non-zero weight identified by the zero-skipping circuit together with a plurality of corresponding values from the input tensor.