# US 2015/0178246 A1 · Processing device for performing convolution operations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0178246 A1
- Document ID: US20150178246A1
- Status: Published application
- Kind: A1
- Application number: 14/136,302
- Filed: 2013-12-20
- Published: 2015-06-25
- Earliest claimed: 2013-12-20
- Assignees: Intel Corporation
- Inventors: Enric Herrero Abellanas; Marc Lupon; Ayose J. Falcon; Frederico C. Pratas; Fernando Latorre; Pedro Lopez
- Web page: https://patentgrep.com/patent/US20150178246A1

## CPC

- G06V 10/82: using neural networks
- G06F 17/15: Correlation function computation {including computation of convolution operations (arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00)}
- G06F 17/153: {Multidimensional correlation or convolution}
- G06F 18/2413: based on distances to training or reference patterns
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06V 10/764: using classification, e.g. of video objects
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods for performing convolution operations. An example processing system comprises: a processing core; and a convolver unit to apply a convolution filter to a plurality of input data elements represented by a two-dimensional array, the convolver unit comprising a plurality of multipliers coupled to two or more sets of latches, wherein each set of latches is to store a plurality of data elements of a respective one-dimensional section of the two-dimensional array.

## Claims

### Claim 1

A processing system, comprising:
a processing core; and
a convolver unit to apply a convolution filter to a plurality of input data elements represented by a two-dimensional array, the convolver unit comprising a plurality of multipliers coupled to two or more sets of latches, wherein each set of latches is to store a plurality of data elements of a respective one-dimensional section of the two-dimensional array.

### Claim 2 (depends on 1)

The processing system of claim 1, wherein the two or more sets of latches are provided by a first set of latches and a second set of latches, wherein the first set of latches is to store a first plurality of data elements of a first one-dimensional section of the two-dimensional array and the second set of latches is to store a second plurality of data elements of a second one-dimensional section of the two-dimensional array.

### Claim 3 (depends on 1)

The processing system of claim 1, wherein the first one-dimensional section is provided by a first row the and second one-dimensional section is provided by a second row that is adjacent to the first row in the two-dimensional array.

### Claim 4 (depends on 1)

The processing system of claim 1, wherein each multiplier of the plurality of multipliers is to apply a convolution filter element to an input data element.

### Claim 5 (depends on 1)

The processing system of claim 1, wherein the convolver unit further comprises a subsampling circuit to perform at least one of: averaging a plurality of convolution results or determining a maximum value of convolution results.

### Claim 6 (depends on 1)

The processing system of claim 1, wherein the convolver unit further comprises a plurality of multiplexers, each multiplexer to select one of: a first latch output, a second latch output, or an external memory input to be supplied to a multiplier.

### Claim 7 (depends on 1)

The processing system of claim 1, wherein the convolution filter is provided by a rectangular matrix of pre-defined binary values; and
wherein each set of latches comprises a number of latches which is equal to a dimension of the convolution filter.

### Claim 8 (depends on 1)

The processing system of claim 1, wherein a latch of the first plurality of latches is connected in series with a latch of a second plurality of latches.

### Claim 9

A method, comprising:
setting, by a processing device, an initial position of an input window in a two-dimensional array of input data elements;
applying a convolution filter to a plurality of input data elements referenced by the input window;
shifting the input window, relatively to its previous position, by one or more positions along a one-dimensional section of the two-dimensional array; and
iteratively repeating the applying and shifting operations for a pre-defined number of times, wherein at least one of the applying operations comprises receiving an input data element from one of: an external memory or an internal latch of two or more sets of latches, each set of latches corresponding to a respective one-dimensional section of the two-dimensional array.

### Claim 10 (depends on 9)

The method of claim 9, wherein the applying and shifting operations comprise:
applying a convolution filter to a first plurality of input data elements referenced by the input window;
shifting the input window, relatively to its previous position, by one position along a first one-dimensional section of the two-dimensional array;
applying the convolution filter to a second plurality of input data elements referenced by the input window;
shifting the input window, relatively to the initial position, by one position along a second one-dimensional section of the two-dimensional array;
applying the convolution filter to a third plurality of input data elements referenced by the input window;
shifting the input window, relatively to its previous position, by one position along a first one-dimensional section of the two-dimensional array; and
applying the convolution filter to a fourth plurality of input data elements referenced by the input window.

### Claim 11 (depends on 10)

The method of claim 10, further comprising: repeating, a pre-defined number of times, at least one of sequences of the shifting and applying operations, wherein the number of times is determined as a pooling sample dimension reduced by one.

### Claim 12 (depends on 9)

The method of claim 9, wherein the first one-dimensional section is provided by a first row the and second one-dimensional section is provided by a second row that is adjacent to the first row in the two-dimensional array.

### Claim 13 (depends on 9)

The method of claim 9, further comprising:
performing a subsampling operation of convolution operation results.

### Claim 14 (depends on 9)

The method of claim 9, wherein the receiving further comprises storing the input data element in an internal latch.

### Claim 15

A system-on-chip (SoC), comprising:
a plurality of convolver units, each convolver unit to apply a convolution filter to a plurality of input data elements represented by a two-dimensional array, each convolver unit comprising a plurality of multipliers coupled to two or more sets of latches, wherein each set of latches is to store a plurality of data elements of a respective one-dimensional section of the two-dimensional array.

### Claim 16 (depends on 15)

The SoC of claim 15, wherein the two or more sets of latches are provided by a first set of latches and a second set of latches, wherein the first set of latches is to store a first plurality of data elements of a first one-dimensional section of the two-dimensional array and the second set of latches is to store a second plurality of data elements of a second one-dimensional section of the two-dimensional array.

### Claim 17 (depends on 15)

The SoC of claim 15, wherein the first one-dimensional section is provided by a first row the and second one-dimensional section is provided by a second row that is adjacent to the first row in the two-dimensional array.

### Claim 18 (depends on 15)

The SoC of claim 15, wherein each multiplier is to apply a convolution filter element to an input data element.

### Claim 19 (depends on 15)

The SoC of claim 15, further comprising a subsampling circuit to perform at least one of: averaging a plurality of convolution results or determining a maximum value of convolution results.

### Claim 20 (depends on 15)

The SoC of claim 15, wherein each convolver unit further comprises a plurality of multiplexers, each multiplexer to select one of: a first latch output, a second latch output, or an external memory input to be supplied to a multiplier.
