Hardware architecture for introducing activation sparsity in neural network

A computer-implemented method for processing a neural network, the computer-implemented method comprising:

• storing, by a memory circuit, a weight tensor and an input activation tensor of a first node of the neural network;
• performing, by a multiply circuit, a convolution operation between the weight tensor and the input activation tensor of the first node to generate a version of an output activation tensor;
• receiving, by a comparator tree circuit including a hierarchical arrangement of comparator circuits, the output activation tensor;
• comparing, by the comparator tree circuit, between at least two of active values in the output activation tensor;
• setting, by the comparator tree circuit, one or more of the active values in the output activation tensor to zero according to the comparison to generate a version of a sparsified output activation tensor with a reduced number of active values relative to the output activation tensor;
• and
• using the sparsified output activation tensor as an input activation tensor of a second node of the neural network.

A computing device, comprising:

• a memory configured to store a neural network that comprises a plurality of nodes, at least one node associated with a weight tensor and an input activation tensor;
• and
• an artificial intelligence accelerator coupled to the memory, the artificial intelligence accelerator configured to:
• store the weight tensor and the input activation tensor that corresponds toa first node of the neural network;
• perform a convolution operation between the weight tensor and the input activation tensor of the first node to generate a version of an output activation tensor;
• compare between at least two of active values in the output activation tensor to set one or more active values in the output activation tensor to zero using a comparator tree circuit that includes a hierarchical arrangement of comparator circuits;
• and
• generate a version of a sparsified output activation tensor with a reduced number of the active values relative to the output activation tensor by setting the one or more active values in the output activation tensor to zero according to the comparison, the sparsified output activation tensor used as an input activation tensor of a second node of the neural network.