Neural processing device and method for pruning thereof
AssigneeRebellions Inc.
InventorJinwook Oh
Life of this patent
Estimating the term…
Abstract
A neural processing device and method for pruning thereof are provided. The neural processing device includes a processing unit configured to perform calculations, an L0 memory configured to store input and output data of the processing unit, wherein the input and output data include a two-dimensional weight matrix and a weight manipulator configured to receive the two-dimensional weight matrix and partition it into preset sizes to thereby generate partitioned matrices, to generate a pruning matrix by pruning the partitioned matrix, and to transmit the pruning matrix to the processing unit.
Claims18
A neural processing device having a neural core, comprising:
- processing circuitry configured to perform deep learning calculation tasks;
- and
- an L0 memory configured to store input and output data of the processing circuitry, wherein the input and output data includes a two-dimensional weight matrix,
- wherein weight manipulator circuitry of the neural core:
- receives the two-dimensional weight matrix as input data signal from the L0 memory;
- checks a width of an SIMD (Single Instruction/Multiple Data) input of a processing element using a width identifier that then generates and sends a confirmation signal to a weight initializer that, upon receiving the confirmation signal, initializes the two-dimensional weight matrix;
- partitions the two-dimensional weight matrix into preset sizes according to the width of the SIMD input of the processing element using a matrix divider that then generates at least one partitioned matrix;
- counts a number of zero groups based on an initial threshold value to generate counting information,
- selects a partitioned matrix of the at least one partitioned matrix having a largest number of zero groups in the counting information as a reference partitioned matrix,
- adjusts at least one threshold value of at least one remaining partitioned matrix of the at least one partitioned matrix other than the reference partitioned matrix to obtain an updated threshold value,
- prunes the at least one partitioned matrix based on the updated threshold value to generate a pruning matrix;
- and
- transmits the pruning matrix as output data signal to the processing circuitry that includes a weight register temporarily storing the pruning matrix and performs an operation using the pruning matrix.
A method for pruning of a neural processing device comprising at least one neural core of the neural processing device including processing circuitry for deep learning calculation tasks performing operations according to stored instructions, the method comprising:
- receiving, by weight manipulator circuitry, a two-dimensional weight matrix as an input data signal;
- checking, by the weight manipulator circuitry, a width of an SIMD input of a processing element;
- initializing, by the weight manipulator circuitry, the two-dimensional weight matrix;
- partitioning, by the weight manipulator circuitry, the two-dimensional weight matrix into at least one partitioned matrix based on the width of the SIMD input, wherein the at least one partitioned matrix contains at least one group;
- counting a number of zero groups based on an initial threshold value to generate counting information;
- selecting a partitioned matrix of the at least one partitioned matrix having a largest number of zero groups in the counting information as a reference partitioned matrix;
- adjusting at least one threshold value of at least one remaining partitioned matrix of the at least one partitioned matrix other than the reference partitioned matrix to generate an updated threshold value;
- pruning the at least one partitioned matrix based on the updated threshold value to generate a pruning matrix, wherein the pruning comprises generating a representative value of a first group and changing the first group to a zero group if the representative value is less than or equal to the updated threshold value;
- transmitting, by the weight manipulator circuitry, the pruning matrix as an output data signal to the processing circuitry that includes a weight register temporarily storing the pruning matrix and performs an operation using the pruning matrix;
- and wherein the two-dimensional weight matrix contains at least one weight, wherein the at least one weight and the updated threshold value are trained via an artificial neural network, and wherein the updated threshold value is trained in a direction in which a number of zero groups is uniformly distributed for each partitioned matrix of the at least one partitioned matrix.
Dependencies are inferred from the claim text.