# US 2025/0156690 A1 · Inline and in-memory two-dimensional convolution and convolutional neural network hardware engines for networking devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0156690 A1
- Document ID: US20250156690A1
- Status: Published application
- Kind: A1
- Application number: 19/025,671
- Filed: 2025-01-16
- Published: 2025-05-15
- Earliest claimed: 2022-07-18
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Francisco FONS LLUIS
- Web page: https://patentgrep.com/patent/US20250156690A1

## CPC

- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A two-dimensional (2D) convolution hardware engine for a networking device includes processing circuitry that further includes multiple processing stages forming a pipeline to perform an inline and in-memory 2D convolution operation on a received dataset. A first processing stage of the pipeline shifts the dataset in a first direction of a 2D space, a second processing stage shifts the dataset in a second direction of the 2D space, a third processing stage calculates a product of each pixel of a kernel of the shifted 2D image by multiplying the pixel with a filter weight of the kernel, and a fourth processing stage calculates a sum of the products of all pixels of the kernel of the shifted 2D image. A convolutional neural network (CNN) computation hardware engine for performing an inline and in-memory CNN computation operation on the received dataset is included.

## Claims

### Claim 1

A Two-Dimensional (2D) convolution hardware engine for a networking device, the 2D convolution hardware engine comprising processing circuitry,
wherein the processing circuitry comprises multiple processing stages connected to form a pipeline and configured to perform an inline and in-memory 2D convolution operation on a received dataset, wherein:
a first processing stage of the pipeline is configured to shift the dataset in a first direction of a 2D space,
a second processing stage of the pipeline is configured to shift the dataset in a second direction of the 2D space,
a third processing stage of the pipeline is configured to calculate a product of each pixel of a kernel of the shifted dataset by multiplying the pixel with a filter weight of the kernel, and
a fourth processing stage of the pipeline is configured to calculate a sum of the products of all pixels of the kernel of the shifted dataset.

### Claim 2 (depends on 1)

The 2D convolution hardware engine according to claim 1, wherein
the dataset is a 2D image.

### Claim 3 (depends on 1)

The 2D convolution hardware engine according to claim 1, wherein
the inline 2D convolution is executed in parallel and/or simultaneously for two or more pixels of the dataset.

### Claim 4 (depends on 1)

The 2D convolution hardware engine according to claim 1, further comprising a first set of configurable registers configured to be programmed with a first set of values, and to be accessible by the processing circuitry at an initialization phase and/or during the inline 2D convolution operation of the 2D convolution hardware engine, wherein the first set of values defines one or more physical parameters related to the inline 2D convolution operation performed on the dataset.

### Claim 5 (depends on 4)

The 2D convolution hardware engine according to claim 4, wherein the first set of configurable registers is configured to be memory mapped.

### Claim 6 (depends on 1)

The 2D convolution hardware engine according to claim 1, further comprising a second set of configurable registers configured to be programmed with a second set of values, and to be accessible by the processing circuitry at the initialization phase and/or during the inline 2D convolution operation of the 2D convolution hardware engine, wherein the second set of values defines one or more functional parameters related to the inline 2D convolution operation performed on the dataset.

### Claim 7 (depends on 6)

The 2D convolution hardware engine according to claim 6, wherein the second set of configurable registers is configured to be memory mapped.

### Claim 8 (depends on 1)

The 2D convolution hardware engine according to claim 1, further comprising:
an ingress port for receiving the dataset; and/or
an egress port for outputting a result of the 2D convolution operation performed on the dataset.

### Claim 9 (depends on 1)

A convolutional neural network (CNN) computation hardware engine configured to perform an inline and in-memory CNN computation operation on a received dataset comprising:
an ingress port for receiving the dataset;
a Two-Dimensional (2D) convolution hardware engine according to claim 1 configured to perform an inline 2D convolution operation on the dataset;
an egress port for outputting a result of the inline 2D convolution operation performed on the dataset; and
a loopback path configured to loop back the result output by the 2D convolution hardware engine as an input to the 2D convolution hardware engine to form a CNN path.

### Claim 10 (depends on 9)

The CNN computation hardware engine according to claim 9, further comprising one or more multiplexers (MUX) configured to select between two or more input datasets to output one selected output dataset.

### Claim 11 (depends on 9)

The CNN computation hardware engine according to claim 9, further comprising one or more dual-port random access memories (RAMs) configured to simultaneously permit one or more reads and one or more writes of one or more datasets.

### Claim 12 (depends on 11)

The CNN computation hardware engine according to claim 11, wherein the one or more dual-port RAMs are either one or more first in first out (FIFO) or simple and/or true dual-port RAM memories configured to queue one or more input datasets and/or one or more output datasets of the 2D convolution hardware engine.

### Claim 13 (depends on 9)

The CNN computation hardware engine according to claim 9, further comprising a pooling processing stage configured to reduce a dimension of the dataset.

### Claim 14 (depends on 9)

The CNN computation hardware engine according to claim 9, further comprising an activation function processing stage configured to calculate a weighted sum of pixels of a kernel input to an activation function processing stage, and add a bias to a weighted sum.

### Claim 15 (depends on 9)

The CNN computation hardware engine according to claim 9, further comprising:
two or more dual-port RAMs comprising at least a first dual-port RAM and a second dual-port RAM, wherein the loopback path is further configured to loop back the output of the 2D convolution hardware engine and/or a pooling processing stage and/or an activation function processing stage to the first dual-port RAM and/or to the second dual-port RAM to form a residual CNN path;
a shift processing stage comprising:
a first shift processing stage configured to shift the dataset in a first direction of a 2D space, and
a second shift processing stage configured to shift the dataset in a second direction of the 2D space; and
an adding processing stage configured to add the result output by the 2D convolution hardware engine to an output of the second dual-port RAM.

### Claim 16 (depends on 9)

The CNN computation hardware engine according to claim 9, further comprising a third set of configurable registers configured to be programmed with a third set of values, and to be accessible by the processing circuitry at an initialization phase and/or during the inline CNN computation operation of the CNN computation hardware engine, wherein the third set of values defines one or more physical parameters related to the inline CNN computation operation performed on the dataset.

### Claim 17 (depends on 16)

The CNN computation hardware engine according to claim 16, wherein the third set of configurable registers is configured to be memory mapped.

### Claim 18 (depends on 9)

The CNN computation hardware engine according to claim 9, further comprising a fourth set of configurable registers configured to be programmed with a fourth set of values, and to be accessible by the processing circuitry at the initialization phase and/or during the inline CNN computation operation of the CNN computation hardware engine, wherein the fourth set of values defines one or more functional parameters related to the inline CNN computation operation performed on the dataset.

### Claim 19 (depends on 18)

The CNN computation hardware engine according to claim 18, wherein the fourth set of configurable registers is configured to be memory mapped.

### Claim 20 (depends on 1)

A networking device configured to receive a dataset as an input and provide a processed dataset as an output, the networking device comprising:
a normalization processing stage, wherein the networking device is configured to perform normalization functions on the received dataset in the normalization processing stage to provide a normalized dataset;
a match processing stage, wherein the networking device is configured to perform filtering and policing functions on the normalized dataset in the match processing stage to provide a filtered and policed dataset; and
an action processing stage configured to perform switching, routing and gatewaying processing functions on the filtered and policed dataset in the action processing stage,
wherein the action processing stage comprises a Two-Dimensional (2D) convolution hardware engine according to claim 1, and wherein the 2D convolution hardware engine is configured to perform inline 2D convolution operations on the filtered and policed dataset in the action processing stage.

### Claim 21 (depends on 20)

The networking device according to claim 20, further comprising a convolutional neural network (CNN) computation hardware engine, wherein the 2D convolution hardware engine is integrated into the CNN computation hardware engine.

### Claim 22

A method for a Two-Dimensional (2D) convolution hardware engine for an inline and in-memory 2D convolution operation on a dataset, the method being performed by the 2D convolution hardware engine and comprising:
receiving the dataset;
shifting the dataset in a first direction of a 2D space;
shifting the dataset in a second direction of the 2D space;
calculating a product of each pixel of an image kernel of the shifted dataset by multiplying the pixel with a filter weight of the image kernel; and
calculating a sum of the products of all pixels of the image kernel of the shifted dataset.

### Claim 23 (depends on 1)

A method for a convolutional neural network (CNN) computation hardware engine for an inline and in-memory CNN computation operation on a dataset, the method being performed by the CNN computation hardware engine and comprising:
receiving the dataset via an ingress port;
performing an inline Two-Dimensional (2D) convolution operation on the dataset by a 2D convolution hardware engine according to claim 1;
outputting a result of the inline 2D convolution operation performed on the dataset by an egress port; and
looping back the result output by the 2D convolution hardware engine to an internal queue of the 2D convolution hardware engine to form a CNN path by means of a loopback path.
