US 2019/0164538 A1 · MEMORY COMPRESSION IN A DEEP NEURAL NETWORK

Abstract

Aspects disclosed in the detailed description include memory compression in a deep neural network (DNN). To support a DNN application, a fully connected weight matrix associated with a hidden layer(s) of the DNN is divided into a plurality of weight blocks to generate a weight block matrix with a first number of rows and a second number of columns. A selected number of weight blocks are randomly designated as active weight blocks in each of the first number of rows and updated exclusively during DNN training. The weight block matrix is compressed to generate a sparsified weight block matrix including exclusively active weight blocks. The second number of columns is compressed to reduce memory footprint and computation power, while the first number of rows is retained to maintain accuracy of the DNN, thus providing the DNN in an efficient hardware implementation without sacrificing accuracy of the DNN application.

Claims20

Dependent claims

A method for reducing memory requirement of a deep neural network (DNN) comprising:

dividing a fully connected weight matrix associated with at least one hidden layer of the DNN into a plurality of weight blocks of a determined block size to generate a weight block matrix having a first number of rows and a second number of columns;
randomly designating a selected number of active weight blocks in each of the first number of rows based on a target sparsity ratio;
updating exclusively the selected number of active weight blocks in each of the first number of rows during a DNN training for a DNN application;
compressing the weight block matrix to generate a sparsified weight block matrix comprising exclusively the selected number of active weight blocks in each of the first number of rows during the DNN training;
and
storing the sparsified weight block matrix for the DNN application upon completion of the DNN training, wherein the sparsified weight block matrix is configured to be used directly for DNN classification.

3 more elements

A non-transitory computer-readable medium comprising software with instructions configured to:

divide a fully connected weight matrix associated with at least one hidden layer of a deep neural network (DNN) into a plurality of weight blocks of a determined block size to generate a weight block matrix having a first number of rows and a second number of columns;
randomly designate a selected number of active weight blocks in each of the first number of rows based on a target sparsity ratio;
update exclusively the selected number of active weight blocks in each of the first number of rows during a DNN training for a DNN application;
compress the weight block matrix to generate a sparsified weight block matrix comprising exclusively the selected number of active weight blocks in each of the first number of rows during the DNN training;
and
store the sparsified weight block matrix for the DNN application upon completion of the DNN training, wherein the sparsified weight block matrix is configured to be used directly for DNN classification.

3 more elements

An electronic device configured to implement a deep neural network (DNN) comprising:

one or more memory banks each configured to store a sparsified weight block matrix generated by a method comprising:
dividing a fully connected weight matrix associated with at least one hidden layer of the DNN into a plurality of weight blocks of a determined block size to generate a weight block matrix having a first number of rows and a second number of columns;
randomly designating a selected number of active weight blocks in each of the first number of rows based on a target sparsity ratio;
updating exclusively the selected number of active weight blocks in each of the first number of rows during a DNN training for a DNN application;
compressing the weight block matrix to generate the sparsified weight block matrix comprising exclusively the selected number of active weight blocks in each of the first number of rows during the DNN training;
and
storing the sparsified weight block matrix in a memory bank among the one or more memory banks for the DNN application upon completion of the DNN training, wherein the sparsified weight block matrix is configured to be used directly for DNN classification;
and
DNN computation circuitry coupled to the one or more memory banks and configured to:
receive a plurality of neuron inputs from a hidden layer in the DNN;
retrieve a selected sparsified weight block matrix corresponding to the hidden layer from a selected memory bank among the one or more memory banks;
and
generate a neuron output based on the plurality of neuron inputs and the selected sparsified weight block matrix.

10 more elements

Dependencies are inferred from the claim text.

Memory compression in a deep neural network

Life of this patent

Memory compression in a deep neural network

Life of this patent

Abstract

Claims20