US 2020/0090030 A1 · INTEGRATED CIRCUIT FOR CONVOLUTION CALCULATION IN DEEP NEURAL NETWORK AND METHOD THEREOF

Abstract

An integrated circuit applied in a deep neural network is disclosed. The integrated circuit comprises at least one processor, a first internal memory, a second internal memory, at least one MAC circuit, a compressor and a decompressor. The processor performs a cuboid convolution over decompression data for each cuboid of an input image fed to any one of multiple convolution layers. The MAC circuit performs multiplication and accumulation operations associated with the cuboid convolution to output a convoluted cuboid. The compressor compresses the convoluted cuboid into one compressed segment and store it in the second internal memory. The decompressor decompresses data from the second internal memory segment by segment to store the decompression data in the first internal memory. The input image is horizontally divided into multiple cuboids with an overlap of at least one row for each channel between any two adjacent cuboids.

Claims19

Dependent claims

A method applied in an integrated circuit for use in a deep neural network, the integrated circuit comprising a first internal memory and a second internal memory, the method comprising:

(a) decompressing a first compressed segment associated with a current cuboid of a first input image and outputted from the first internal memory to store decompressed data in the second internal memory;
(b) performing cuboid convolution over the decompressed data to generate a 3D pointwise output array;
(c) compressing the 3D pointwise output array into a second compressed segment to store it in the first internal memory;
(d) repeating steps (a) to (c) until all the cuboids associated with a target convolution layer are processed;
and
(e) repeating steps (a) to (d) until all of multiple convolution layers are completed;
wherein the first input image is fed to any one of the convolution layers and horizontally divided into a plurality of cuboids of the same dimension, with an overlap of at least one row for each channel between any two adjacent cuboids;
and
wherein the cuboid convolution comprises a depthwise convolution followed by a pointwise convolution.

6 more elements

An integrated circuit applied in a deep neural network, comprising:

at least one processor configured to perform a cuboid convolution over decompression data for each cuboid of a first input image fed to any one of multiple convolution layers;
a first internal memory coupled to the at least one processor;
at least one multiply-accumulator (MAC) circuit coupled to the at least one processor and the first internal memory for performing multiplication and accumulation operations associated with the cuboid convolution to output a first convoluted cuboid;
a second internal memory for storing multiple compressed segments only;
a compressor coupled to the at least one processor, the at least one MAC circuit and the first and the second internal memories and configured to compress the first convoluted cuboid into one compressed segment to store it in the second internal memory;
and
a decompressor coupled to the at least one processor, the first and the second internal memories and configured to decompress the compressed segments from the second internal memory on a compressed segment by compressed segment basis to store the decompression data for a single cuboid in the first internal memory;
wherein the first input image is horizontally divided into a plurality of cuboids of the same dimension, with an overlap of at least one row for each channel between any two adjacent cuboids;
and
wherein the cuboid convolution comprises a depthwise convolution followed by a pointwise convolution.

7 more elements

Dependencies are inferred from the claim text.

Integrated circuit for convolution calculation in deep neural network and method thereof

Life of this patent

Integrated circuit for convolution calculation in deep neural network and method thereof

Life of this patent

Abstract

Claims19