Neural processing device and method for pruning thereof
AssigneeRebellions Inc.
InventorJinwook Oh
Life of this patent
Estimating the term…
Abstract
A neural processing device and method for pruning thereof are provided. The neural processing device includes a processing unit configured to perform calculations, an L0 memory configured to store input and output data of the processing unit, wherein the input and output data include a two-dimensional weight matrix and a weight manipulator configured to receive the two-dimensional weight matrix and partition it into preset sizes to thereby generate partitioned matrices, to generate a pruning matrix by pruning the partitioned matrix, and to transmit the pruning matrix to the processing unit.
Claims18
A neural processing device having a neural core, comprising:
- processing circuitry configured to perform deep learning calculation tasks;
- and
- an L0 memory configured to store input data and output data of the processing circuitry, wherein the input and output data includes a two-dimensional weight matrix,
- wherein weight manipulator circuitry of the neural core:
- receives the two-dimensional weight matrix from the L0 memory;
- checks a width of an SIMD (Single Instruction/Multiple Data) input of a processing element that then generates and sends a confirmation signal to a weight initializer circuitry that, upon receiving the confirmation signal, initializes the two-dimensional weight matrix;
- partitions the two-dimensional weight matrix into preset sizes according to a width of the SIMD input of the processing element that then generates at least one partitioned matrix;
- counts a number of zero groups in the at least one partitioned matrix based on an initial threshold value to generate counting information;
- selects a partitioned matrix of the at least one partitioned matrix having a largest number of zero groups in the counting information as a reference partitioned matrix;
- adjusts at least one threshold value of at least one remaining partitioned matrix of the at least one partitioned matrix other than the reference partitioned matrix to obtain an updated threshold value;
- prunes the at least one partitioned matrix by using the updated threshold value for controlling the number of zero groups included in each of the at least one partitioned matrix to generate a pruning matrix;
- and
- transmits the pruning matrix as output data signal tothe processing circuitry.
A method for pruning of a neural processing device comprising at least one neural core of the neural processing device including processing circuitry for deep learning calculation tasks performing operations according to stored instructions, the method comprising:
- receiving, by weight manipulator circuitry, a two-dimensional weight matrix;
- checking, by the weight manipulator circuitry, a width of an SIMD (Single Instruction/Multiple Data) input of a processing element that then generates and sends a confirmation signalto a weight initializer circuitry that, upon receiving the confirmation signal, initializes the two-dimensional weight matrix;
- partitioning, by the weight manipulator circuitry, the two-dimensional weight matrix into at least one partitioned matrix based on a width of the SIMD input;
- counting a number of zero groups in the at least one partitioned matrix based on an initial threshold value to generate counting information;
- selecting a partitioned matrix of the at least one partitioned matrix having a largest number of zero groups in the counting information as a reference partitioned matrix;
- adjusting at least one threshold value of at least one remaining partitioned matrix of the at least one partitioned matrix other than the reference partitioned matrix to generate an updated threshold value;
- pruning the at least one partitioned matrix by using the updated threshold value for controlling the number of zero groups included in each of the at least one partitioned matrix to generate a pruning matrix;
- and
- transmitting, by the weight manipulator circuitry, the pruning matrix as an output data signal to the processing circuitry.
Dependencies are inferred from the claim text.