Contiguous sparsity pattern neural networks

A system comprising:

• one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
• receiving one or more parameter matrices of a neural network, wherein at least a first parameter matrix has parameter values stored in a contiguous sparsity pattern, the contiguous sparsity pattern being a two-dimensional pattern composed of zero values and a plurality of non-square rectangular blocks of non-zero values, the number of zero values being greater than the number of zero-values, wherein the neural network has been trained in a training process that produces the contiguous sparsity pattern of the parameter values of the first parameter matrix;
• storing the first parameter matrix in storage associated with a computing device;
• receiving an input vector of activation values;
• and
• performing, by the computing device, an inference pass of the neural network to generate an output vector, including performing operations comprising:
• reading, from the storage associated with the computing device, one or more activation values from the input vector,
• reading, from the storage associated with the computing device, a block of non-zero parameter values, and multiplying each of the one or more activation values by one or more of the block of non-zero parameter values.

One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• receiving one or more parameter matrices of a neural network, wherein at least a first parameter matrix has parameter values stored in a contiguous sparsity pattern, the contiguous sparsity pattern being a two-dimensional pattern composed of zero values and a plurality of non-square rectangular blocks of non-zero values, the number of zero values being greater than the number of zero-values, wherein the neural network has been trained in a training process that comprises partitioning the first parameter matrix into a plurality of non-square blocks and assigning zero or another reserved value to one or more of the non-square blocks of parameter values following a determination that magnitudes of the parameter values of the one or more blocks satisfy a threshold criterion;
• storing the first parameter matrix in storage associated with a computing device;
• receiving an input vector of activation values;
• and
• performing, by the computing device, an inference pass of the neural network to generate an output vector, including performing operations comprising:
• reading, from the storage associated with the computing device, one or more activation values from the input vector,
• reading, from the storage associated with the computing device, a block of non-zero parameter values, and
• multiplying each of the one or more activation values by one or more of the block of non-zero parameter values.