# US 12,217,184 B2 · Low-power, high-performance artificial neural network training accelerator and acceleration method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,184 B2
- Document ID: US12217184B2
- Status: Granted
- Kind: B2
- Application number: 17/317,900
- Filed: 2021-05-12
- Published: 2022-07-14
- Granted: 2025-02-04
- Earliest claimed: 2021-01-11
- Assignees: Korea Advanced Institute of Science and Technology
- Applicants: Korea Advanced Institute of Science and Technology
- Inventors: Hoi Jun Yoo; Sang Yeob Kim
- Web page: https://patentgrep.com/patent/US12217184B2

## CPC

- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 15/80: comprising an array of processing units with common control, e.g. single instruction multiple data processors (G06F15/82 takes precedence {; for correlation function computation G06F17/15})
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

A method of accelerating training of a low-power, high-performance artificial neural network (ANN) includes (a) performing fine-grained pruning and coarse-grained pruning to generate sparsity in weights by a pruning unit in a convolution core of a cluster in a lower-power, high-performance ANN trainer; (b) selecting and performing dual zero skipping according to input sparsity, output sparsity, and the sparsity of weights by the convolution core, and (c) restricting access to a weight memory during training by allowing a deep neural network (DNN) computation core and a weight pruning core to share weights retrieved from a memory by the convolution core.

## Claims

### Claim 1

A method of accelerating training of a low-power, high-performance artificial neural network (ANN), the method comprising:
(a) performing fine-grained pruning and coarse-grained pruning to generate sparsity in weights by a pruning unit in a convolution core of a cluster in a lower-power, high-performance ANN trainer, wherein
the fine-grained pruning generates a random sparsity pattern by replacing values with small magnitudes with zeros, and
the coarse-grained pruning calculates similarities between weights or magnitudes of the weights on an output channel basis and replaces similar consecutive weights or consecutive weights with the small magnitudes with consecutive zeros;
(b) selecting and performing dual zero skipping according to input sparsity, output sparsity, and the sparsity of weights by the convolution core,
wherein, when the convolution core performs the dual zero skipping by using the sparsity of weights, the convolution core skips zeroes in weight data by
skipping computations using the consecutive zeros caused by the coarse-grained pruning at once, and
skipping computations using random zeroes caused by the fine-grained pruning one at a time; and
(c) restricting access to a weight memory during training by allowing a deep neural network (DNN) computation core and a weight pruning core to share weights retrieved from a memory by the convolution core.

### Claim 2 (depends on 1)

The method according to claim 1, wherein in the step of (a), the pruning unit maximizes the generated sparsity with the same accuracy by periodically applying the coarse-grained pruning and the fine-grained pruning to a model requiring training during the training, instead of applying the coarse-grained pruning and the fine-grained pruning once to a pre-trained model.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the pruning unit determines a periodicity of the coarse-grained pruning and the fine-grained pruning by recording a maximum accuracy achieved during training until before pruning and when a higher accuracy than the maximum accuracy score is achieved by training again after the pruning, performing next pruning.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the pruning unit controls a threshold set for pruning in each period to decrease an additionally generated sparsity ratio for an input channel with high weight sparsity generated by the coarse-grained pruning and the fine-grained pruning and increase the additionally generated sparsity ratio for an input channel with low weight sparsity generated by the coarse-grained pruning and the fine-grained pruning.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the pruning unit performs pruning by coarse-grained similarity comparison, coarse-grained magnitude comparison, or fine-grained magnitude comparison, and
wherein the pruning unit performs the pruning, while changing the threshold by a specific fixed value in each period, until no further accuracy recovery is possible by the training after the pruning.

### Claim 6 (depends on 1)

The method according to claim 1, wherein in the step of (b), the convolution core receives a sparsity pattern for two of inputs, outputs, and weights, and performs zero skipping convolution.

### Claim 7

A low-power, high-performance artificial neural network (ANN) training accelerator comprising:
a plurality of clusters, each including a plurality of convolution cores,
wherein a pruning unit in each of the convolution cores performs fine-grained pruning and coarse-grained pruning to generate sparsity in weights, and the convolution core selects and performs dual zero skipping according to input sparsity, output sparsity, and the sparsity of weights and restricts access to a weight memory during training by allowing a deep neural network (DNN) computation core and a weight pruning core to share weights retrieved from a memory, and
wherein the pruning unit performs the fine-grained pruning for generating a random sparsity pattern by replacing values with small magnitude with zeros, and performs the coarse-grained pruning by calculating similarities or magnitudes of weights on an output channel basis, and replacing consecutive similar weights or consecutive weights with small magnitudes with consecutive zeros,
wherein, when each convolution core performs zero skipping convolution, the convolution core skips zeroes in weight data by
skipping computations using random zeroes caused by the fine-grained pruning one at a time, and
skipping computations using the consecutive zeros caused by the coarse-grained pruning at once.