# US 2021/0240483 A1 · Memory apparatus with a built-in logic operation unit and operation method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0240483 A1
- Document ID: US20210240483A1
- Status: Published application
- Kind: A1
- Application number: 17/163,587
- Filed: 2021-02-01
- Published: 2021-08-05
- Earliest claimed: 2020-02-03
- Assignees: Winbond Electronics Corp.
- Applicants: Winbond Electronics Corp.
- Inventors: Chien-Ti Hou; Wu-Chuan Cheng
- Web page: https://patentgrep.com/patent/US20210240483A1

## CPC

- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 9/3893: {controlled in tandem, e.g. multiplier-accumulator}
- G06F 9/30101: {Special purpose registers}
- G06F 9/30189: {according to execution mode, e.g. mode flag}
- G06F 9/3557: {using program counter as base address}
- G06F 9/544: {Buffers; Shared memory; Pipes}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06F 2207/4824: Neural networks
- G06N 3/045: Combinations of networks
- G06N 3/063: using electronic means
- G06N 3/08: Learning methods
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A memory apparatus and an operation method thereof are provided. The memory apparatus includes a mode configuration register, a system memory array, a pointer and an arithmetic circuit including logic operation units. The mode configuration register stores weight matrix information and a base address. The system memory array stores feature values in a feature map from the base address according to the weight matrix information. The pointer stores the base address and a weight matrix size to provide pointer information. The arithmetic circuit sequentially or parallelly reads the feature values according to the pointer information. The arithmetic circuit parallelly arranges weight coefficients of a selected weight matrix and the corresponding feature values in each of the corresponding logic operation units according to the weight matrix information, and causes the logic operation units to perform computing operations parallelly to output intermediate layer feature values to an external processing unit.

## Claims

### Claim 1

A memory apparatus adapted for artificial intelligence operation, the memory apparatus comprising:
a mode configuration register configured to store weight matrix information and a base address;
a system memory array configured to store a plurality of feature values in a feature map from the base address according to the weight matrix information;
a pointer coupled to the mode configuration register and the system memory array, and configured to receive and store the base address and a weight matrix size to provide pointer information; and
an arithmetic circuit coupled to the mode configuration register and the pointer, and configured to sequentially or parallelly read the feature values according to the pointer information,
wherein the arithmetic circuit comprises a plurality of logic operation units, and the arithmetic circuit is further configured to parallelly arrange a plurality of weight coefficients of a selected weight matrix and the corresponding feature values in each of the logic operation units according to the weight matrix information, and the logic operation units are configured to perform computing operations parallelly to generate a plurality of intermediate layer feature values, and the arithmetic circuit is further configured to output the intermediate layer feature values to an external processing unit.

### Claim 2 (depends on 1)

The memory apparatus according to claim 1, wherein each of the logic operation units comprises a multiply-accumulate unit, a first buffer, a second buffer and a third buffer, and
the arithmetic circuit is configured to arrange the weight coefficients in the first buffer and arrange the corresponding feature values in the second buffer according to the weight matrix information, so as to provide the corresponding intermediate layer feature values in the third buffer.

### Claim 3 (depends on 1)

The memory apparatus according to claim 1, wherein the mode configuration register further stores a stride setting, and
the pointer comprises:
a pointer register receiving and storing the base address; and
a data length register receiving and storing a data length calculated based on the weight matrix size,
wherein the pointer is configured to update the base address stored in the pointer register according to the base address currently stored in the pointer register, the weight matrix information and the stride setting to provide an updated pointer information, so as to slide in the feature map according to the stride setting.

### Claim 4 (depends on 1)

The memory apparatus according to claim 1, wherein the memory apparatus further comprises:
a vector array coupled to the mode configuration register and the arithmetic circuit, and configured to store a plurality of weight matrices,
wherein the vector array selects the selected weight matrix from the weight matrices according to the weight matrix information to provide the selected weight matrix to the arithmetic circuit.

### Claim 5 (depends on 1)

The memory apparatus according to claim 1, wherein the arithmetic circuit further comprises:
a logic operation configuration unit configured to divide the logic operation units into a plurality of groups according to the weight matrix information, so as to control the logic operation units in a same group to perform the operation parallelly and to control the logic operation units in different groups to perform the operation sequentially.

### Claim 6 (depends on 1)

The memory apparatus according to claim 1, wherein the mode configuration register further stores retention information, and
the memory apparatus further comprises:
a feedback control unit coupled to the mode configuration register, the system memory array and the arithmetic circuit, and to store the intermediate layer feature values back to the system memory array according to the retention information as new feature values.

### Claim 7 (depends on 6)

The memory apparatus according to claim 6, wherein the retention information comprises the number of times for which the intermediate layer feature values are stored back to the system memory array, and
the feedback control unit comprises a counter configured to control a transfer gate to store the intermediate layer feature values back to the system memory array according to the number of times.

### Claim 8 (depends on 3)

The memory apparatus according to claim 3, wherein the pointer is configured to determine whether a sum of the base address and the data length reaches a final address used by the system memory array to store the feature map,
when the sum of the base address and the data length reaches the final address, the arithmetic circuit outputs the intermediate layer feature values to the processing unit, and
when the sum of the base address and the data length is not equal to the final address, the pointer updates the pointer information.

### Claim 9 (depends on 1)

The memory apparatus according to claim 1, wherein each of the logic operation units comprises an arithmetic logic unit, a first buffer, a second buffer and a third buffer, and
the arithmetic circuit is configured to arrange the weight coefficients in the first buffer and arranges the corresponding feature values in the second buffer according to the weight matrix information, so as to provide the corresponding intermediate layer feature values in the third buffer.

### Claim 10 (depends on 1)

The memory apparatus according to claim 1, wherein each of the logic operation units comprises a floating point unit, a first buffer, a second buffer and a third buffer, and
the arithmetic circuit is configured to arrange the weight coefficients in the first buffer and arranges the corresponding feature values in the second buffer according to the weight matrix information, so as to provide the corresponding intermediate layer feature values in the third buffer.

### Claim 11

An operation method of a memory apparatus, adapted to the memory apparatus comprising a system memory array, a pointer and a plurality of logic operation units, the operation method comprising:
storing weight matrix information and a base address;
storing a plurality of feature values in a feature map from the base address according to the weight matrix information in the system memory array;
storing the base address and a weight matrix size in the pointer to provide pointer information;
reading the feature values sequentially or parallelly according to the pointer information;
arranging a plurality of weight coefficients in a selected weight matrix and the corresponding feature values parallelly in each of the corresponding logic operation units according to the weight matrix information;
performing computing operations parallelly by the logic operation units to generate a plurality of intermediate layer feature values; and
outputting the intermediate layer feature values to an external processing unit.

### Claim 12 (depends on 11)

The operation method of the memory apparatus according to claim 11, wherein each of the logic operation units comprises a multiply-accumulate unit, a first buffer, a second buffer and a third buffer, and
arranging the weight coefficients in the selected weight matrix and the corresponding feature values parallelly in each of the corresponding logic operation units according to the weight matrix information comprises:
arranging the weight coefficients in the first buffer and arranging the corresponding feature values in the second buffer according to the weight matrix information, so as to provide the corresponding intermediate layer feature values in the third buffer.

### Claim 13 (depends on 11)

The operation method of the memory apparatus according to claim 11, wherein the pointer further comprises a pointer register, and the operation method further comprises:
calculating a data length based on the weight matrix size;
updating the base address stored in the pointer register according to the base address currently stored in the pointer register, the weight matrix information and a stride setting to provide an updated pointer information, so as to slide in the feature map according to the stride setting.

### Claim 14 (depends on 11)

The operation method of the memory apparatus according to claim 11, further comprising:
selecting the selected weight matrix from a plurality of weight matrices according to the weight matrix information to provide the selected weight matrix to the logic operation units.

### Claim 15 (depends on 11)

The operation method of the memory apparatus according to claim 11, wherein performing the operation parallelly by the logic operation units comprises:
dividing the logic operation units into a plurality of groups according to the weight matrix information;
controlling the logic operation units in a same group to perform the operation parallelly; and
controlling the logic operation units in different groups to perform the operation sequentially.

### Claim 16 (depends on 11)

The operation method of the memory apparatus according to claim 11, further comprising:
storing the intermediate layer feature values according to retention information as new feature values.

### Claim 17 (depends on 16)

The operation method of the memory apparatus according to claim 16, wherein the retention information comprises the number of times for which the intermediate layer feature values are stored, and
wherein storing the intermediate layer feature values according to the retention information as the new feature values comprises:
storing the intermediate layer feature values according to the number of times.

### Claim 18 (depends on 13)

The operation method of the memory apparatus according to claim 13, wherein after performing the operation parallelly by the logic operation units to generate the intermediate layer feature values, the operation method further comprises:
determining whether a sum of the base address and the data length reaches a final address used to store the feature map;
when the sum of the base address and the data length reaches the final address, outputting the intermediate layer feature values to the processing unit; and
when the sum of the base address and the data length is not equal to the final address, updating the pointer information.
