# US 2019/0171690 A1 · Max pooling in a matrix processing architecture

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171690 A1
- Document ID: US20190171690A1
- Status: Published application
- Kind: A1
- Application number: 16/266,847
- Filed: 2019-02-04
- Published: 2019-06-06
- Earliest claimed: 2016-12-30
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Horace Lau; Tony L. Werner
- Web page: https://patentgrep.com/patent/US20190171690A1

## CPC

- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06V 10/82: using neural networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

In one embodiment, an apparatus comprises a multi-dimensional memory and a plurality of processing elements to perform a matrix operation, wherein the matrix operation comprises a max pooling operation on one or more matrix operands. The plurality of processing elements comprises one or more matrix processors, and the plurality of processing elements is configured to: receive matrix data from the multi-dimensional memory, wherein the matrix data is associated with the one or more matrix operands; extract the one or more matrix operands from the matrix data; perform the max pooling operation using the one or more matrix operands; and obtain a result of the max pooling operation.

## Claims

### Claim at position 1

1-25. (canceled)

### Claim 26

A matrix processing circuit, comprising:
a plurality of memory resource blocks;
control circuitry to cause the matrix processing circuit to perform a backward pooling operation, wherein the backward pooling operation is to generate, based on a pooled matrix operand, a partially reconstructed matrix corresponding to an original matrix operand, wherein the pooled matrix operand corresponds to an output of a forward pooling operation performed on the original matrix operand;
read circuitry to receive the pooled matrix operand from one or more of the plurality of memory resource blocks;
slice circuitry to:
extract a plurality of value-index pairs from the pooled matrix operand; and
for each value-index pair of the plurality of value-index pairs:
generate, based on the value-index pair, a corresponding portion of the partially reconstructed matrix;
identify a completed portion of the partially reconstructed matrix to write to memory, wherein the completed portion will not be modified during generation of a remaining portion of the partially reconstructed matrix; and
write the completed portion of the partially reconstructed matrix to one or more of the plurality of memory resource blocks; and
output circuitry to output the partially reconstructed matrix as a result of the backward pooling operation.

### Claim 27 (depends on 26)

The matrix processing circuit of claim 26, further comprising a first in first out (FIFO) memory to store a sliding window of elements associated with the partially reconstructed matrix.

### Claim 28 (depends on 27)

The matrix processing circuit of claim 27, wherein the completed portion of the partially reconstructed matrix comprises one or more elements at a front of the FIFO memory.

### Claim 29 (depends on 28)

The matrix processing circuit of claim 28, wherein a number of elements within the completed portion is based on a stride parameter of the backward pooling operation.

### Claim 30 (depends on 27)

The matrix processing circuit of claim 27, wherein the FIFO memory comprises one or more status bits to track whether each element in the sliding window of elements has been modified.

### Claim 31 (depends on 26)

The matrix processing circuit of claim 26, wherein the forward pooling operation is to generate the pooled matrix operand based on the original matrix operand, wherein the pooled matrix operand is smaller in size than the original matrix operand.

### Claim 32 (depends on 31)

The matrix processing circuit of claim 31, wherein the forward pooling operation comprises a max pooling operation.

### Claim 33 (depends on 26)

The matrix processing circuit of claim 26, wherein:
the forward pooling operation is associated with forward propagation in an artificial neural network; and
the backward pooling operation is associated with backward propagation in the artificial neural network.

### Claim 34

A system, comprising:
a processor to execute a plurality of instructions associated with an application, wherein the plurality of instructions comprises one or more instructions associated with a backward pooling operation, wherein the backward pooling operation is to generate, based on a pooled matrix operand, a partially reconstructed matrix corresponding to an original matrix operand, wherein the pooled matrix operand corresponds to an output of a forward pooling operation performed on the original matrix operand; and
matrix processing circuitry to perform the backward pooling operation, wherein the matrix processing circuitry comprises:
a plurality of memory resource blocks; and
circuitry to:
receive the pooled matrix operand from one or more of the plurality of memory resource blocks;
extract a plurality of value-index pairs from the pooled matrix operand;
for each value-index pair of the plurality of value-index pairs:
generate, based on the value-index pair, a corresponding portion of the partially reconstructed matrix;
identify a completed portion of the partially reconstructed matrix to write to memory, wherein the completed portion will not be modified during generation of a remaining portion of the partially reconstructed matrix; and
write the completed portion of the partially reconstructed matrix to one or more of the plurality of memory resource blocks; and
output the partially reconstructed matrix as a result of the backward pooling operation.

### Claim 35 (depends on 34)

The system of claim 34, wherein the matrix processing circuitry further comprises a first in first out (FIFO) memory to store a sliding window of elements associated with the partially reconstructed matrix.

### Claim 36 (depends on 35)

The system of claim 35, wherein the completed portion of the partially reconstructed matrix comprises one or more elements at a front of the FIFO memory.

### Claim 37 (depends on 36)

The system of claim 36, wherein a number of elements within the completed portion is based on a stride parameter of the backward pooling operation.

### Claim 38 (depends on 35)

The system of claim 35, wherein the FIFO memory comprises one or more status bits to track whether each element in the sliding window of elements has been modified.

### Claim 39 (depends on 34)

The system of claim 34, wherein the forward pooling operation is to generate the pooled matrix operand based on the original matrix operand, wherein the pooled matrix operand is smaller in size than the original matrix operand.

### Claim 40 (depends on 39)

The system of claim 39, wherein the forward pooling operation comprises a max pooling operation.

### Claim 41

At least one non-transitory machine accessible storage medium having instructions stored thereon, the instructions, when executed on a machine, cause the machine to:
receive a command to perform a backward pooling operation, wherein the backward pooling operation is to generate, based on a pooled matrix operand, a partially reconstructed matrix corresponding to an original matrix operand, wherein the pooled matrix operand corresponds to an output of a forward pooling operation performed on the original matrix operand;
receive the pooled matrix operand from one or more of a plurality of memory resource blocks;
extract a plurality of value-index pairs from the pooled matrix operand;
for each value-index pair of the plurality of value-index pairs:
generate, based on the value-index pair, a corresponding portion of the partially reconstructed matrix;
identify a completed portion of the partially reconstructed matrix to write to memory, wherein the completed portion will not be modified during generation of a remaining portion of the partially reconstructed matrix; and
write the completed portion of the partially reconstructed matrix to one or more of the plurality of memory resource blocks; and
output the partially reconstructed matrix as a result of the backward pooling operation.

### Claim 42 (depends on 41)

The storage medium of claim 41, wherein the instructions further cause the machine to store, on a first in first out (FIFO) memory, a sliding window of elements associated with the partially reconstructed matrix.

### Claim 43 (depends on 42)

The storage medium of claim 42, wherein the completed portion of the partially reconstructed matrix comprises one or more elements at a front of the FIFO memory.

### Claim 44 (depends on 43)

The storage medium of claim 43, wherein a number of elements within the completed portion is based on a stride parameter of the backward pooling operation.

### Claim 45 (depends on 43)

The storage medium of claim 43, wherein the instructions further cause the machine to determine whether the one or more elements at the front of the FIFO memory have been modified, wherein the FIFO memory comprises one or more status bits to track whether each element in the sliding window of elements has been modified.

### Claim 46

A method, comprising:
receiving a command to perform a backward pooling operation, wherein the backward pooling operation is to generate, based on a pooled matrix operand, a partially reconstructed matrix corresponding to an original matrix operand, wherein the pooled matrix operand corresponds to an output of a forward pooling operation performed on the original matrix operand;
receiving the pooled matrix operand from one or more of a plurality of memory resource blocks;
extracting a plurality of value-index pairs from the pooled matrix operand;
for each value-index pair of the plurality of value-index pairs:
generating, based on the value-index pair, a corresponding portion of the partially reconstructed matrix;
identifying a completed portion of the partially reconstructed matrix to write to memory, wherein the completed portion will not be modified during generation of a remaining portion of the partially reconstructed matrix; and
writing the completed portion of the partially reconstructed matrix to one or more of the plurality of memory resource blocks; and
outputting the partially reconstructed matrix as a result of the backward pooling operation.

### Claim 47 (depends on 46)

The method of claim 46, further comprising storing, on a first in first out (FIFO) memory, a sliding window of elements associated with the partially reconstructed matrix.

### Claim 48 (depends on 47)

The method of claim 47, wherein the completed portion of the partially reconstructed matrix comprises one or more elements at a front of the FIFO memory.

### Claim 49 (depends on 48)

The method of claim 48, wherein a number of elements within the completed portion is based on a stride parameter of the backward pooling operation.

### Claim 50 (depends on 48)

The method of claim 48, further comprising determining whether the one or more elements at the front of the FIFO memory have been modified, wherein the FIFO memory comprises one or more status bits to track whether each element in the sliding window of elements has been modified.