Methods and devices for irregular pruning for automatic speech recognition

An apparatus for automatic speech recognition, comprising:

• one or more processors;
• and
• a memory configured to store instructions executable by the one or more processors;
• wherein the one or more processors, upon execution of the instructions, are configured to:
• receive, by a first layer in a neural network that is executed on the one or more processors, a first input associated with one or more audio feature sequences;
• generate a weight matrix for a layer of a plurality of layers in the neural network, wherein the weight matrix comprises a set of weights associated with the layer, the plurality of layers comprises the first layer receiving a first, and the plurality of layers are executed on the one or more processors;
• divide the weight matrix into a plurality of blocks, each block comprising a plurality of weights, wherein each block has an identical dimension and the identical dimension of each block that is obtained by dividing the weight matrix is determined according to a size of a tensor core unit of the one or more processors;
• determine a pre-determined percentage for each block, wherein the pre-determined percentage is X %, X is a positive integer;
• select, based on the pre-determined percentage, a sub-set of weights from the plurality of weights in each block for block-wise pruning, wherein a first ratio of a number of the sub-set of weights in each block to a number of the plurality of weights in each block is X %, wherein the sub-set of weights comprise at least one weight;
• set the sub-set of weights in each block to zero to generate a block-wise pruned weight matrix, wherein other weights in each block remain non-zero, wherein a second ratio of a number of the other weights in each block to the number of the plurality of weights in each block is 1-X %;
• and
• generating a final output for the one or more audio feature sequences based on the block-wise pruned weight matrix.

A non-transitory computer readable storage medium, comprising instructions stored therein, wherein, upon execution of the instructions by one or more processors, the instructions cause the one or more processors to perform acts comprising:

• receiving, by a first layer in a neural network that is executed on the one or more processors, a first input associated with one or more audio feature sequences;
• generating a weight matrix for a layer of a plurality of layers in the neural network, wherein the weight matrix comprises a set of weights associated with the layer, the plurality of layers comprises the first layer receiving the first input, and the plurality of layers are executed on the one or more processors;
• dividing the weight matrix into a plurality of blocks, each block comprising a plurality of weights, wherein each block has an identical dimension and the identical dimension of each block that is obtained by dividing the weight matrix is determined according to a size of a tensor core unit of the one or more processors;
• determining a pre-determined percentage for each block, wherein the pre-determined percentage is X %, X is a positive integer;
• selecting, based on the pre-determined percentage, a sub-set of weights from the plurality of weights in each block for block-wise pruning, wherein a first ratio of a number of the sub-set of weights in each block to a number of the plurality of weights in each block is X %, wherein the sub-set of weights comprise at least one weight;
• setting the sub-set of weights in each block to zero to generate a block-wise pruned weight matrix, wherein other weights in each block remain non-zero, wherein a second ratio of a number of the other weights in each block to the number of the plurality of weights in each block is 1-X %;
• and
• generating a final output for the one or more audio feature sequences based on the block-wise pruned weight matrix.