Performing average pooling in hardware

A system comprising:

• a hardware circuit;
• and
• one or more storage devices storing instructions that are operable, when executed by the hardware circuit, to cause the hardware circuit to perform operations to generate a layer output tensor that is equivalent to an output of an average pooling neural network layer of a neural network, by performing operations comprising:
• sending an input tensor to the average pooling neural network layer from a first portion of memory of the hardware circuit to a hardware matrix computation unit of the hardware circuit;
• sending a kernel from a second portion of memory of the hardware circuit to the hardware matrix computation unit, wherein the kernel has a size equal to a window of the average pooling neural network layer and is composed of elements that are each an identity matrix;
• performing, by a hardware matrix computation unit of the hardware circuit, a convolution of an input tensor to the average pooling neural network layer and the kernel with a size equal to the window of the average pooling neural network layer and composed of elements that are each an identity matrix to generate a first tensor, wherein at least part of the convolution is performed without having to pass data off the hardware circuit, thereby allowing for an inference of the average pooling neural network layer of the neural network to be determined without modifying the hardware circuit and avoiding processing delays resulting from performing part of the convolution off the hardware circuit;
• performing, by the hardware matrix computation unit, operations to cause each element of the first tensor to be divided by a number of elements in the window of the average pooling neural network layer to generate an initial output tensor;
• and
• rescaling, by the hardware matrix computation unit, edge and corner elements of the initial output tensor by performing element-wise multiplication of the initial output tensor with a rescaling tensor, wherein the rescaling tensor includes a first scaling factor for rescaling the edge elements and a second scaling factor for rescaling the corner elements.

A method for processing an average pooling neural network layer of a neural network on a hardware circuit, the method comprising:

• receiving a request to process the average pooling neural network layer on the hardware circuit;
• and
• in response, generating instructions that when executed by a hardware matrix computation unit of the hardware circuit, cause the hardware matrix computation unit to generate a layer output tensor that is equivalent to an output of the average pooling neural network layer by performing operations comprising:
• sending an input tensor to the average pooling neural network layer from a first portion of memory of the hardware circuit to the hardware matrix computation unit;
• sending a kernel from a second portion of memory of the hardware circuit to the hardware matrix computation unit, wherein the kernel has a size equal to a window of the average pooling neural network layer and is composed of elements that are each an identity matrix;
• performing, by the hardware matrix computation unit, a convolution of an input tensor to the average pooling neural network layer and the kernel with a size equal to the window of the average pooling neural network layer and composed of elements that are each an identity matrix to generate a first tensor, wherein at least part of the convolution is performed without having to pass data off the hardware circuit, thereby allowing for an inference of the average pooling neural network layer of the neural network to be determined without modifying the hardware circuit and avoiding processing delays resulting from performing part of the convolution off the hardware circuit;
• rescaling, by the hardware matrix computation unit, edge and corner elements of the first tensor by performing element-wise multiplication of the first tensor with a rescaling tensor to generate an initial output tensor;
• and
• performing, by the hardware matrix computation unit, operations to cause each element of the initial output tensor to be divided by a number of elements in the window of the average pooling neural network layer to generate the layer output tensor, wherein the rescaling tensor includes a first scaling factor for rescaling the edge elements and a second scaling factor for rescaling the corner elements.