# US 2021/0241080 A1 · Artificial intelligence accelerator and operation thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241080 A1
- Document ID: US20210241080A1
- Status: Published application
- Kind: A1
- Application number: 16/782,972
- Filed: 2020-02-05
- Published: 2021-08-05
- Earliest claimed: 2020-02-05
- Assignees: Macronix International Co., Ltd.
- Applicants: MACRONIX International Co., Ltd.
- Inventors: HANG-TING LUE; Teng-Hao Yeh; Po-Kai Hsu; Ming-Liang Wei
- Web page: https://patentgrep.com/patent/US20210241080A1

## CPC

- G06N 3/063: using electronic means
- G06F 5/01: for shifting, e.g. justifying, scaling, normalising {(digital stores in which the information is moved stepwise, e.g. shift-registers G11C19/00; digital stores in which the information circulates G11C21/00)}
- G06F 7/50: Adding; Subtracting (G06F7/483 - G06F7/491, G06F7/544 - G06F7/556 take precedence)
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06F 2207/4824: Neural networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An artificial intelligence accelerator receives a binary input data set and a selected layer of layers of overall weight pattern. The artificial intelligence accelerator includes processing tiles and a summation output circuit. Each processing tile receives one of input data subsets of the input data set and performs a convolution operation on weight blocks of each sub weight pattern of the overall weight pattern to obtain weight operation values and then obtains a weight output value expected from a direct convolution operation on the input data subset with the sub weight pattern through performing a multistage shifting and adding operation on the weight operation values. The summation output circuit sums up the plurality of weight output values through a multistage shifting and adding operation, so as to obtain a sum value expected from a direct convolution operation performed on the input data set with the overall weight pattern.

## Claims

### Claim 1

An artificial intelligence accelerator, configured to receive a binary input data set and a selected layer of a plurality of layers of an overall weight pattern to perform a convolution operation, wherein the input data set is divided into a plurality of data subsets, and the artificial intelligence accelerator comprises:
a plurality of processing tiles, wherein each of the processing tiles comprises:
a receive-end component, configured to receive one of the data subsets;
a weight storage unit, configured to store a part of the overall weight pattern, wherein the partial weight storage unit comprises a plurality of weight blocks, and each of the weight blocks stores a block part of the partial weight pattern in order of bits, wherein a cell array structure of the weight storage unit, with respect to a corresponding one of the data sets, configured to perform a convolution operation on the data subset with each block part respectively to obtain a plurality of sequential weight operation values; and
a block-wise output circuit, comprising a plurality of shifters and a plurality of adders, and configured to sum up the plurality of weight operation values through a multistage shifting and adding operation, so as to obtain a weight output value expected from a direct convolution operation performed on the data subset with the partial weight pattern; and
a summation output circuit, comprising a plurality of shifters and a plurality of adders, and configured to sum up the plurality of weight output values through a multistage shifting and adding operation, so as to obtain a sum value expected from a direct convolution operation performed on the input data set with the overall weight pattern.

### Claim 2 (depends on 1)

The artificial intelligence accelerator according to claim 1, wherein the input data set comprises i bits, and is divided into p data subsets, i and p are integers, and each of the data subsets comprises i/p bits.

### Claim 3 (depends on 1)

The artificial intelligence accelerator according to claim 1, wherein the input data set comprises i bits, and the quantity of the plurality of processing tiles is p, the input data set is divided into p data subsets, i and p are integers greater than or equal to 2, i is greater than p, and each of the data subsets comprises i/p bits.

### Claim 4 (depends on 3)

The artificial intelligence accelerator according to claim 3, wherein the quantity of the plurality of weight blocks comprised in the partial weight storage unit is q, q is an integer greater than or equal to 2, the partial weight storage unit comprises j bits, j and q are integers greater than or equal to 2, j is greater than q, and each of the weight blocks comprises j/q memory cells.

### Claim 5 (depends on 4)

The artificial intelligence accelerator according to claim 4, wherein the block-wise output circuit comprises at least one shifter and at least one adder in each stage of the shifting and adding operation; and
two adjacent input values of a plurality of input values in each stage are one processing unit, after passing through the shifter, one input value in a higher bit location is added by the adder to the other input value in a lower bit location, and is output to a next stage, and in a last stage, a single value is output and used as the weight output value corresponding to the processing tile.

### Claim 6 (depends on 5)

The artificial intelligence accelerator according to claim 5, wherein a shift amount of the shifter in a first stage is j/q memory cells, and a shift amount of the shifter in a next stage is twice that of the shifter in a previous stage.

### Claim 7 (depends on 4)

The artificial intelligence accelerator according to claim 4, wherein the summation output circuit comprises at least one shifter and at least one adder in each stage of the shifting and adding operation; and
two adjacent input values of a plurality of input values in each stage are one processing unit, after passing through the shifter, one input value in a higher bit location is added by the adder to the other input value in a lower bit location, and is output to a next stage, and in a last stage, a single value is output and used as the sum value.

### Claim 8 (depends on 7)

The artificial intelligence accelerator according to claim 7, wherein a shift amount of the shifter in a first stage is i/p bits, and a shift amount of the shifter in a next stage is twice that of the shifter in a previous stage.

### Claim 9 (depends on 1)

The artificial intelligence accelerator according to claim 1, further comprising:
a normalization processing circuit, configured to normalize the sum value to obtain a normalization sum value; and
a quantization processing circuit, configured to quantize the normalization sum value into an integer value by using a base number.

### Claim 10 (depends on 1)

The artificial intelligence accelerator according to claim 1, wherein the processing circuit comprises a plurality of sense amplifiers, respectively configured to sense each block part to perform a convolution operation to obtain a plurality of sensed values as the plurality of weight operation values.

### Claim 11

A processing method applied to an artificial intelligence accelerator, wherein the artificial intelligence accelerator receives a binary input data set and a selected layer of a plurality of layers of an overall weight pattern to perform a convolution operation, and the input data set is divided into a plurality of data subsets, and the processing method comprises:
using a plurality of processing tiles, and each of the processing tiles comprises operations of:
using a receive-end component to receive one of the data subsets;
using a weight storage unit to store a part of the overall weight pattern, wherein the partial weight storage unit comprises a plurality of weight blocks, and each of the weight blocks stores a block part of the partial weight pattern in order of bits, wherein a cell array structure of the weight storage unit, with respect to a corresponding one of the data sets, is configured
to perform a convolution operation on the data subset with each block part respectively to obtain a plurality of sequential weight operation values;
using a block-wise output circuit that comprises a plurality of shifters and a plurality of adders to sum up the plurality of weight operation values through a multistage shifting and adding operation, so as to obtain a weight output value expected from a direct convolution operation performed on the data subset with the partial weight pattern; and
using a summation output circuit that comprises a plurality of shifters and a plurality of adders to sum up the plurality of weight output values through a multistage shifting and adding operation, so as to obtain a sum value expected from a direct convolution operation performed on the input data set with the overall weight pattern.

### Claim 12 (depends on 11)

The processing method of the artificial intelligence accelerator according to claim 11, wherein the input data set comprises i bits, and is divided into p data subsets, i and p are integers, and each of the data subsets comprises i/p bits.

### Claim 13 (depends on 11)

The processing method of the artificial intelligence accelerator according to claim 11, wherein the input data set comprises i bits, and the quantity of the plurality of processing tiles is p, the input data set is divided into p data subsets, i and p are integers greater than or equal to 2, i is greater than p, and each of the data subsets comprises i/p bits.

### Claim 14 (depends on 13)

The processing method of the artificial intelligence accelerator according to claim 13, wherein the quantity of the plurality of weight blocks comprised in the partial weight storage unit is q, q is an integer greater than or equal to 2, the partial storage memory unit comprises j bits, j and q are integers greater than or equal to 2, j is greater than q, and each of the weight blocks comprises j/q memory cells.

### Claim 15 (depends on 14)

The processing method of the artificial intelligence accelerator according to claim 14, wherein each stage of the shifting and adding operation performed by the block-wise output circuit comprises using at least one shifter and at least one adder; and
two adjacent input values of a plurality of input values in each stage are one processing unit, after passing through the shifter, one input value in a higher bit location is added by the adder to the other input value in a lower bit location, and is output to a next stage, and in a last stage, a single value is output and used as the weight output value corresponding to the processing tile.

### Claim 16 (depends on 15)

The processing method of the artificial intelligence accelerator according to claim 15, wherein a shift amount of the shifter in a first stage is j/q memory cells, and a shift amount of the shifter in a next stage is twice that of the shifter in a previous stage.

### Claim 17 (depends on 14)

The processing method of the artificial intelligence accelerator according to claim 14, wherein each stage of the shifting and adding operation performed by the summation output circuit comprises using at least one shifter and at least one adder; and
two adjacent input values of a plurality of input values in each stage are one processing unit, after passing through the shifter, one input value in a higher bit location is added by the adder to the other input value in a lower bit location, and is output to a next stage, and in a last stage, a single value is output and used as the sum value.

### Claim 18 (depends on 17)

The processing method of the artificial intelligence accelerator according to claim 17, wherein a shift amount of the shifter in a first stage is i/p bits, and a shift amount of the shifter in a next stage is twice that of the shifter in a previous stage.

### Claim 19 (depends on 11)

The processing method of the artificial intelligence accelerator according to claim 11, further comprising:
using a normalization processing circuit to normalize the sum value to obtain a normalization sum value; and
using a quantization processing circuit to quantize the normalization sum value into an integer value by using a base number.

### Claim 20 (depends on 11)

The processing method of the artificial intelligence accelerator according to claim 11, wherein the processing circuit comprises a plurality of sense amplifiers, respectively configured to sense each block part to perform a convolution operation to obtain a plurality of sensed values as the plurality of weight operation values.
