Convolution acceleration with embedded vector decompression
AssigneesSTMicroelectronics S.r.l., STMicroelectronics International N.V.
InventorsThomas Boesch, Giuseppe Desoli, Surinder Pal Singh, Carmine Cappetta
Life of this patent
Estimating the term…
Abstract
Techniques and systems are provided for implementing a convolutional neural network. One or more convolution accelerators are provided that each include a feature line buffer memory, a kernel buffer memory, and a plurality of multiply-accumulate (MAC) circuits arranged to multiply and accumulate data. In a first operational mode the convolutional accelerator stores feature data in the feature line buffer memory and stores kernel data in the kernel data buffer memory. In a second mode of operation, the convolutional accelerator stores kernel decompression tables in the feature line buffer memory.
Claims32
A system for implementing a convolutional neural network (CNN), the system comprising:
- a plurality of convolution accelerators, each of the plurality of convolution accelerators including:
- a feature line buffer memory;
- a kernel buffer memory;
- and
- a multiply-accumulate (MAC) cluster including a plurality of MAC circuits coupled to the feature line buffer memory and to the kernel buffer memory, and which, in operation, multiply and accumulate received feature data and kernel data,
- wherein, each of the plurality of convolution accelerators performs first operations in a convolutional acceleration mode in which the feature line buffer memory stores feature data, and second operations in a fully connected acceleration mode in which the feature line buffer memory stores kernel decompression tables.
A convolution accelerator comprising:
- a feature line buffer memory;
- a kernel buffer memory;
- and
- a multiply-accumulate (MAC) cluster coupled to the feature line buffer memory and to the kernel buffer memory and comprising a plurality of MAC circuits, wherein the plurality of MAC circuits, in operation, multiply and accumulate feature data and kernel data,
- wherein, in operation, the convolution accelerator performs first operations in a first mode in which the feature line buffer memory stores feature data, and second operations in a second mode in which the feature line buffer memory stores kernel decompression tables, wherein the first mode is a convolutional acceleration mode and wherein the second mode is a fully connected acceleration mode.
A method, comprising:
- operating a convolutional accelerator having a kernel data buffer memory, a feature line buffer memory, and a multiply-accumulate (MAC) cluster having a plurality of MAC circuits, in a first operational mode, the operating the convolutional accelerator in the first operational mode including:
- storing feature data in the feature line buffer memory;
- storing kernel data in the kernel data buffer memory;
- and
- performing MAC operations using feature line data stored in the feature line buffer memory, and kernel data stored in the kernel data buffer memory;
- and
- operating the convolutional accelerator in a second operational mode, the operating the convolutional accelerator in the second operational mode including:
- storing kernel decompression tables in the feature line buffer memory;
- decompressing compressed kernel data using kernel decompression tables stored in the feature line buffer memory, generating decompressed kernel data;
- storing the decompressed kernel data in the kernel data buffer memory;
- and
- performing MAC operations using feature line data, and decompressed kernel data stored in the kernel data buffer memory.
A non-transitory computer-readable medium having contents that, when executed by one or more hardware processors of a convolution accelerator, cause the one or more hardware processors to perform a method, the method comprising:
- operating the convolutional accelerator in a first operational mode, the convolutional accelerator having a kernel data buffer memory, a feature line buffer memory, and a multiply-accumulate (MAC) cluster having a plurality of MAC circuits, the operating the convolutional accelerator in the first operational mode including:
- storing feature data in the feature line buffer memory;
- storing kernel data in the kernel data buffer memory;
- and
- performing MAC operations using feature line data stored in the feature line buffer memory, and kernel data stored in the kernel data buffer memory;
- and
- operating the convolutional accelerator in a second operational mode, the operating the convolutional accelerator in the second operational mode including:
- storing kernel decompression tables in the feature line buffer memory;
- decompressing compressed kernel data using kernel decompression tables stored in the feature line buffer memory, generating decompressed kernel data;
- storing the decompressed kernel data in the kernel data buffer memory;
- and
- performing MAC operations using feature line data, and decompressed kernel data stored in the kernel data buffer memory.
Dependencies are inferred from the claim text.