Machine learning accelerator mechanism

An apparatus comprising:

• at least one processor to perform operations to implement a neural network and to perform convolution operations and mean and variance operations for the neural network;
• and
• accelerator circuitry communicatively coupled to the at least one processor, the accelerator circuitry to:
• receive, from the at least one processor, input matrix data for a batch normalization layer of the neural network;
• store the input matrix data in a memory of the accelerator circuitry;
• perform, as part of operations of the batch normalization layer, fused computation of the convolution operations and the mean and variance operations for the input matrix data, wherein the fused computation of the convolution operations and the mean and variance operations performs the convolution operations and the mean and variance operations together with a same set of hardware circuitry;
• forward results of the fused computation to a normalization operation of the accelerator circuitry;
• compute the normalization operation on the results, weight gradients by multiplying the transpose of the input matrix data with the weight matrix;
• and
• output a normalization result of the normalization operation from the batch normalization layer.

A method comprising:

• receiving, by accelerator circuitry from at least one processor that implements a neural network, input matrix data for a batch normalization layer of the neural network, wherein the batch normalization layer is to perform convolution operations and mean and variance operations for the neural network;
• storing, by the accelerator circuitry, the input matrix data in a memory of the accelerator circuitry;
• performing, by the accelerator circuitry as part of operations for the batch normalization layer, fused computation of convolution operations and mean and variance operations for the input matrix data, wherein the fused computation of the convolution operations and the mean and variance operations performs the convolution operations and the mean and variance operations together with a same set of hardware circuitry;
• forwarding, by the accelerator circuitry, results of the fused computation to a normalization operation of the accelerator circuitry;
• computing, by the accelerator circuitry, the normalization operation on the results, weight gradients by multiplying the transpose of the input matrix data with the weight matrix;
• and
• outputting, by the accelerator circuitry, a normalization result of the normalization operation from the batch normalization layer.

A non-transitory computer-readable medium having stored thereon executable computer program instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• receiving, by accelerator circuitry from at least one processor of the one or more processors that implements a neural network, input matrix data for a batch normalization layer of the neural network, wherein the batch normalization layer is to perform convolution operations and mean and variance operations for the neural network;
• storing, by the accelerator circuitry, the input matrix data in a memory of the accelerator circuitry;
• performing, by the accelerator circuitry as part of operations for the batch normalization layer, fused computation of convolution operations and mean and variance operations for the input matrix data, wherein the fused computation of the convolution operations and the mean and variance operations performs the convolution operations and the mean and variance operations together with a same set of hardware circuitry;
• forwarding, by the accelerator circuitry, results of the fused computation to a normalization operation of the accelerator circuitry;
• computing, by the accelerator circuitry, the normalization operation on the results, weight gradients by multiplying the transpose of the input matrix data with the weight matrix;
• and
• outputting, by the accelerator circuitry, a normalization result of the normalization operation from the batch normalization layer.