US 2022/0188638 A1 · DATA REUSE IN DEEP LEARNING

Abstract

An apparatus for convolution operations is provided. The apparatus includes a PE array, a datastore, writing modules, reading modules, and a controlling module. The PE array performs MAC operations. The datastore includes databanks, each of which stores data to be used by a column of the PE array. The writing modules transfer data from a memory to the datastore. The reading modules transfer data from the datastore to the PE array. Each reading module may transfer data to a particular column of the PE array. The controlling module can determine the rounds of a convolution operation. Each round includes MAC operations based on a weight. The controlling module controls the writing modules and reading modules so that the same data in a databank can be reused in multiple rounds. For different rounds, the controlling module can provide a reading module accesses to different databanks.

Claims25

Dependent claims

An apparatus for deep learning includes:

a processing element array, comprising a plurality of processing elements arranged in columns and rows, the processing element array configured to perform a convolution operation including a sequence of rounds, wherein each round includes multiply-accumulate (MAC) operations based on a different weight in a weight matrix;
a datastore comprising databanks that are configured to store data to be used by the processing element array for the convolution operation;
a writing module configured to write data stored in a memory into the datastore;
a plurality of reading modules, a reading module of the plurality of reading modules corresponding to a column of the processing element array and configured to transfer data from a databank of the datastore to the column of the processing element array in a round of the convolution operation;
and
a controlling module configured to provide the reading module accesses to different databanks of the datastore in different rounds of the convolution operation for transferring data from the different databanks to the column of the processing element array in the different rounds.

3 more elements

A method, comprising:

determining a sequence of rounds in a convolution operation based on a weight matrix, each round including multiply-accumulate (MAC) operations performed by a processing element array based on a different weight in the weight matrix, the processing element array comprising columns and rows of processing elements;
for a first round in the sequence:
providing a first reading module an access to a first databank of a datastore, the first reading module configured to transfer data in the first databank to a first column of the processing element array, the datastore comprising databanks that are configured to store data that the processing element array uses to perform the convolution operation, and
providing a second reading module an access to a second databank of the datastore, the second reading module configured to transfer data in the second databank to a second column of the processing element array;
and
for a second round that is subsequent to the first round in the sequence of rounds, providing the first reading module an access to the second databank, the first reading module configured to transfer the data in the second databank to the first column of the processing element array.

3 more elements

One or more non-transitory computer-readable media storing instructions executable to perform operations, the operations comprising:

determining a sequence of rounds in a convolution operation based on a weight matrix, each round including multiply-accumulate (MAC) operations performed by a processing element array based on a different weight in the weight matrix, the processing element array comprising columns and rows of processing elements;
for a first round in the sequence:
providing a first reading module an access to a first databank of a datastore, the first reading module configured to transfer data in the first databank to a first column of the processing element array, the datastore comprising databanks that are configured to store data that the processing element array uses to perform the convolution operation, and
providing a second reading module an access to a second databank of the datastore, the second reading module configured to transfer data in the second databank to a second column of the processing element array;
and
for a second round that is subsequent to the first round in the sequence of rounds, providing the first reading module an access to the second databank, the first reading module configured to transfer the data in the second databank to the first column of the processing element array.

3 more elements

Dependencies are inferred from the claim text.

Data reuse in deep learning

Life of this patent

Data reuse in deep learning

Life of this patent

Abstract

Claims25