# US 2022/0012303 A1 · Apparatus and method for matrix multiplication using processing-in-memory

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0012303 A1
- Document ID: US20220012303A1
- Status: Published application
- Kind: A1
- Application number: 17/037,633
- Filed: 2020-09-29
- Published: 2022-01-13
- Earliest claimed: 2020-07-07
- Assignees: Neonexus Pte. Ltd.
- Applicants: NeoNexus Pte. Ltd.
- Inventors: Qilin Zheng
- Web page: https://patentgrep.com/patent/US20220012303A1

## CPC

- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 9/5061: {Partitioning or combining of resources}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

Embodiments of apparatus and method for matrix multiplication using processing-in-memory (PIM) are disclosed. In an example, an apparatus for matrix multiplication includes an array of PIM blocks in rows and columns, a controller, and an accumulator. Each PIM block is configured into a computing mode or a memory mode. The controller is configured to divide the array of PIM blocks into a first set of PIM blocks each configured into the memory mode and a second set of PIM blocks each configured into the computing mode. The first set of PIM blocks are configured to store a first matrix, and the second set of PIM blocks are configured to store a second matrix and calculate partial sums of a third matrix based on the first and second matrices. The accumulator is configured to output the third matrix based on the partial sums of the third matrix.

## Claims

### Claim 1

An apparatus for matrix multiplication, comprising:
an array of processing-in-memory (PIM) blocks in rows and columns, each of which is configured into a computing mode or a memory mode;
a controller configured to divide the array of PIM blocks into a first set of PIM blocks each configured into the memory mode and a second set of PIM blocks each configured into the computing mode, wherein the first set of PIM blocks are configured to store a first matrix, and the second set of PIM blocks are configured to store a second matrix and calculate partial sums of a third matrix based on the first and second matrices; and
an accumulator configured to output the third matrix based on the partial sums of the third matrix.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the first set of PIM blocks consists of a row of the array of PIM blocks, and the second set of PIM blocks consists of a remainder of the array of PIM blocks.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein dimensions of the first set of PIM blocks are smaller than dimensions of the first matrix, and the controller is configured to map the first matrix to the first set of PIM blocks such that the first set of PIM blocks store part of the first matrix at a same time based on the dimensions of the first set of PIM blocks.

### Claim 4 (depends on 3)

The apparatus of claim 3, further comprising a multiplexer (MUX) between the first and second sets of PIM blocks, wherein the controller is further configured to control the MUX to direct data from each PIM block of the first set of PIM blocks to the PIM blocks in a corresponding column of the second set of PIM blocks.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein at least one PIM block of the first set of PIM blocks is not in the same corresponding column.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein dimensions of the second set of PIM blocks match dimensions of the second matrix.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein the second matrix has three or more dimensions, and the controller is configured to map the second matrix to the second set of PIM blocks such that each PIM block of the second set of PIM blocks stores a respective portion of the second matrix.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the controller is configured to control the second set of PIM blocks to perform convolution of the first and second matrices in calculating the partial sums of the third matrix.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the first matrix comprises a feature map in a convolution neural network (CNN), and the second matrix comprises a kernel of the CNN.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein each PIM block in the array of PIM blocks comprises a memory array and a vector-vector multiplication (VVM) engine configured to be disabled in the memory mode.

### Claim 11

A processing-in-memory (PIM) device, comprising:
a memory array configured to store a first vector;
a vector-vector multiplication (VVM) engine; and
a control circuit configured to:
in a computing mode, enable the VVM engine and control the VVM engine to perform a dot product between the first vector and a second vector to generate a partial sum; and
in a memory mode, disable the VVM engine and control the memory array to write in or read out the first vector.

### Claim 12 (depends on 11)

The PIM device of claim 11, wherein the VVM engine comprises a bit counter, a shift accumulator, and a plurality of AND gates.

### Claim 13 (depends on 11)

The PIM device of claim 11, further comprising a first buffer configured to receive and buffer the second vector from another PIM device.

### Claim 14 (depends on 13)

The PIM device of claim 13, further comprising a second buffer configured to buffer and send the partial sum to another PIM device.

### Claim 15

A method for matrix multiplication implemented by an array of processing-in-memory (PIM) blocks in rows and columns, the method comprising:
configuring, by a controller, a first set of PIM blocks of the array of PIM blocks each into a memory mode, and a second set of PIM blocks of the array of PIM blocks each into a computing mode;
mapping, by the controller, a first matrix to the first set of PIM blocks, and a second matrix to the second set of PIM blocks;
calculating, by the second set of PIM blocks, partial sums of a third matrix based on the first and second matrices; and
generating the third matrix based on the partial sums of the third matrix.

### Claim 16 (depends on 15)

The method of claim 15, wherein the first set of PIM blocks consists of a row of the array of PIM blocks, and the second set of PIM blocks consists of a remainder of the array of PIM blocks.

### Claim 17 (depends on 16)

The method of claim 16, wherein dimensions of the first set of PIM blocks are smaller than dimensions of the first matrix, and the method further comprises storing, by the first set of PIM blocks, part of the first matrix at a same time based on the dimensions of the first set of PIM blocks.

### Claim 18 (depends on 17)

The method of claim 17, further comprising directing, by a multiplexer (MUX) between the first and second sets of PIM blocks, data from each PIM block of the first set of PIM blocks to the PIM blocks in a corresponding column of the second set of PIM blocks.

### Claim 19 (depends on 15)

The method of claim 15, wherein dimensions of the second set of PIM blocks match dimensions of the second matrix.

### Claim 20 (depends on 19)

The method of claim 19, wherein the second matrix has three or more dimensions, and the method further comprises storing, by each PIM block of the second set of PIM blocks, a respective portion of the second matrix.
