# US 11,443,505 B2 · Methods and systems for analyzing images in convolutional neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,443,505 B2
- Document ID: US11443505B2
- Status: Granted
- Kind: B2
- Application number: 16/898,972
- Filed: 2020-06-11
- Published: 2020-09-24
- Granted: 2022-09-13
- Earliest claimed: 2016-12-12
- Assignees: Texas Instruments Incorporated
- Applicants: Texas Instruments Incorporated
- Inventors: Mihir Narendra Mody; Manu Mathew; Chaitanya Satish Ghone
- Web page: https://patentgrep.com/patent/US11443505B2

## CPC

- G06V 10/431: {Frequency domain transformation; Autocorrelation}
- G06F 18/24133: {Distances to prototypes}
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method for analyzing images to generate a plurality of output features includes receiving input features of the image and performing Fourier transforms on each input feature. Kernels having coefficients of a plurality of trained features are received and on-the-fly Fourier transforms (OTF-FTs) are performed on the coefficients in the kernels. The output of each Fourier transform and each OTF-FT are multiplied together to generate a plurality of products and each of the products are added to produce one sum for each output feature. Two-dimensional inverse Fourier transforms are performed on each sum.

## Claims

### Claim 1

A system comprising:
a Fourier transform (FT) system module in a neural network configured to generate a transformed input feature by performing a FT on an input feature of a received image;
an on-the-fly Fourier transform (OTF-FT) system module in the neural network configured to generate transformed coefficients by performing an OTF-FT on coefficients; and
a multiplication system module in the neural network, the multiplication system module configured to perform multiplication on the transformed input feature and the transformed coefficients.

### Claim 2 (depends on 1)

The system of claim 1, further comprising pruning system modules for reducing a number of the transformed coefficients generated by the OTF-FT system module.

### Claim 3 (depends on 2)

The system of claim 2, further comprising expansion system modules for expanding the coefficients generated by the pruning system modules.

### Claim 4 (depends on 1)

The system of claim 1, wherein the input feature of the received image is a color plane of the received image.

### Claim 5 (depends on 1)

The system of claim 1, wherein the OTF-FT system module comprises:
a first system module for performing a one-dimensional horizontal direct Fourier transform (1D-HDFT); and
a second system module for performing a one-dimensional vertical direct Fourier transform (1D-VDFT) on a result of the 1D-HDFT.

### Claim 6 (depends on 1)

The system of claim 1, wherein the OTF-FT comprises:
a first system module for performing a one-dimensional horizontal direct Fourier transform (1D-HDFT) on one half of the coefficients in a kernel;
a second system module for splitting results of the 1D-HDFT into a real component and an imaginary component;
a third system module for performing a one-dimensional vertical direct Fourier transform (1D-VDFT) on the real component;
a fourth system module for performing a 1D-VDFT on the imaginary component; and
a fifth system module for merging the real component and the imaginary component.

### Claim 7 (depends on 6)

The system of claim 6, further comprising a sixth system module for expanding results of the merging to generate an array equal in size to the kernel.

### Claim 8 (depends on 1)

The system of claim 1, wherein the multiplication is pointwise multiplication.

### Claim 9 (depends on 1)

The system of claim 1, further comprising a system module for pruning the transformed input feature.

### Claim 10 (depends on 1)

The system of claim 1, further comprising an addition system module configured to perform addition on the multiplied transformed input feature and the multiplied transformed coefficients.

### Claim 11

A method comprising:
receiving an image, the image comprising an input feature;
performing, by a Fourier transform (FT) system module in a neural network, an FT on the input feature to generate a transformed input feature;
performing, by an on-the-fly Fourier transform (OTF-FT) system module in the neural network, an OTF-FT on coefficients to generate transformed coefficients; and
multiplying, by a multiplication system module in the neural network, the transformed input feature and the transformed coefficients.

### Claim 12 (depends on 11)

The method of claim 11, wherein the input feature is a color plane of the image.

### Claim 13 (depends on 11)

The method of claim 11, further comprising pruning the transformed input feature.

### Claim 14 (depends on 11)

The method of claim 11, wherein performing the OTF-FTs comprises:
performing a one-dimensional horizontal direct Fourier transform (1D-HDFT); and
performing a one-dimensional vertical direct Fourier transform (1D-VDFT) on a result of the 1D-HDFT.

### Claim 15 (depends on 11)

The method of claim 11, wherein performing OTF-FTs comprises:
performing a one-dimensional horizontal direct Fourier transform (1D-HDFT) on a portion of the coefficients;
splitting results of the 1D-HDFT into a real component and an imaginary component;
performing a one-dimensional vertical direct Fourier transform (1D-VDFT) on the real component;
performing a 1D-VDFT on the imaginary component; and
merging the real component and the imaginary component.

### Claim 16 (depends on 15)

The method of claim 15, further comprising expanding results of the merging to generate an array equal in size to the coefficients.

### Claim 17 (depends on 15)

The method of claim 15, further comprising pruning the coefficients prior to performing the 1D-HDFT.

### Claim 18 (depends on 11)

The method of claim 11, wherein the multiplication is pointwise multiplication.

### Claim 19

A method comprising:
performing, by a convolutional neural network, a Fourier transform (FT) on an input feature of an image to generate a transformed input feature;
performing, by the convolutional neural network, an on-the-fly Fourier transform (OTF-FT) on coefficients to generate transformed coefficients; and
multiplying, by the convolutional neural network, the transformed input feature and the transformed coefficients.

### Claim 20 (depends on 19)

The method of claim 19, wherein performing the OTF-FTs comprises:
performing a one-dimensional horizontal direct Fourier transform (1D-HDFT); and
performing a one-dimensional vertical direct Fourier transform (1D-VDFT) on a result of the 1D-HDFT.