Accelerated sparse neural network training using transposable pruning masks

A method, comprising:

• receiving a trained deep neural network (DNN) having multiple layers represented by respective weight matrices;
• generating a pruned DNN by, for at least one layer represented by a weight matrix comprising multiple weights, converting the weight matrix into a structured sparse weight matrix:
• finding an optimal pruning mask that (i) specifies a respective subset of the weights in the weight matrix that are to be nulled, (ii) meets a predefined pruning ratio by ensuring at least N weights out of every M successive weights are nulled, and (iii) optimizes a predefined norm,
• nulling the subset of the weights specified in the optimal pruning mask by changing a value of each weight in the subset to zero, and
• increasing one or more absolute values of one or more non-zero weights in the weight matrix based on the mean of the subset of the weights;
• and
• performing an artificial intelligence task using the pruned DNN, wherein performing the artificial intelligence task comprises performing a matrix multiplication in the pruned DNN using the structured sparse weight matrix.

A method, comprising:

• receiving a trained deep neural network (DNN) having multiple layers represented by respective weight matrices;
• generating a pruned DNN by, for at least one layer represented by a weight matrix comprising multiple weights, converting the weight matrix into a structured sparse weight matrix by:
• defining multiple M-element line-blocks in the weight matrix, each line-block comprising the weights along an M-element portion of a row of the weight matrix,
• defining multiple M-element column-blocks in a transpose of the weight matrix, each column-block comprising the weights along an M-element portion of the column of the transpose of the weight matrix,
• specifying a subset of the weights in the weight matrix that are to be nulled, such that (i) at least N weights in each M-element line-block of the weight matrix are to be nulled, and (ii) at least N weights in each M-element, column-block of the transpose of the weight matrix are to be nulled,
• nulling the weights in the subset by changing a value of each weight in the subset to zero, and
• increasing one or more absolute values of one or more non-zero weights in the weight matrix based on the mean of the subset of the weights;
• and
• performing a deep learning operation using the pruned DNN.

An apparatus, comprising:

• a memory configured to store data of a trained deep neural network open (DNN) having multiple layers representative by respective weight matrices;
• and
• a processor configured to:
• generate a pruned DNN by, for at least one layer represented by a weight matrix comprising multiple weights, converting the weight matrix into a structured sparse weight matrix by:
• defining in the weight matrix multiple line-blocks, each line-block comprising the weights along a portion of a row of the weight matrix, and
• in each line-block, nulling a predefined subset of the weights while preserving a mean of the weights of the line-block by absorbing the mean of the predefined subset of the weights into remaining non-zero weights of the line-block, wherein each nulled weight is zero, wherein absorbing the mean of the predefined subset of the weights into the remaining non-zero weights of the line-block comprises increasing one or more absolute values of the remaining non-zero weights of the line-block based on the mean of the predefined subset of the weights, and
• perform an artificial intelligence task using the pruned DNN, wherein performing the artificial intelligence task comprises performing a matrix multiplication in a DNN using the structured sparse weight matrix.

An apparatus, comprising:

• a memory configured to store data of a trained deep neural network (DNN) having multiple layers represented by respective weight matrices;
• and
• a processor, which is configured to:
• generate a pruned DNN by, for at least one layer represented by a weight matrix comprising multiple weights, converting the weight matrix into a structured sparse weight matrix by:
• finding an optimal pruning mask that (i) specifies a respective subset of the weights in the weight matrix that are being nulled, (ii) meets a predefined pruning ratio by ensuring at least N weights out of every M successive weights are nulled, and (iii) optimizes a predefined norm,
• nulling the subset of the weights specified in the optimal pruning mask by changing a value of each weight in the subset to zero, and
• increasing one or more absolute values of one or more non-zero weights in the weight matrix based on the mean of the subset of the weights, and
• perform an artificial intelligence task using the pruned DNN, wherein performing the artificial intelligence task comprises performing a matrix multiplication in the DNN using the structured sparse weight matrix.

An apparatus, comprising:

• a memory configured to store data of a trained deep neural network (DNN) having multiple layers represented by respective weight matrices;
• and
• a processor, which is configured to:
• generate a pruned DNN by, for at least one layer represented by a weight matrix comprising multiple weights, converting the weight matrix into a structured sparse weight matrix by:
• defining multiple M-element line-blocks in the weight matrix, each line-block comprising the weights along an M-element portion of a row of the weight matrix,
• defining multiple M-element column-blocks in a transpose of the weight matrix, each column-block comprising the weights along an M-element portion of a column of the transpose of the weight matrix,
• specifying a subset of the weights in the weight matrix that are to be nulled, such that (i) at least N weights in each M-element line-block of the weight matrix are to be nulled, and (ii) at least N weights in each M-element column-block of the transpose of the weight matrix are to be nulled,
• nulling the weights in the subset by changing a value of each weight in the subset to zero, and
• increasing one or more absolute values of one or more non-zero weights in the weight matrix based on the mean of the subset of the weights, and
• perform a deep learning operation using the pruned DNN.