# US 2022/0036243 A1 · Apparatus with accelerated machine learning processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036243 A1
- Document ID: US20220036243A1
- Status: Published application
- Kind: A1
- Application number: 17/147,858
- Filed: 2021-01-13
- Published: 2022-02-03
- Earliest claimed: 2020-07-29
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Saptarsi Das; Sabitha Kusuma; Arnab Roy; Ankur Deshwal; Kiran Kolar Chandrasekharan; Sehwan Lee
- Web page: https://patentgrep.com/patent/US20220036243A1

## CPC

- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06F 9/3001: {Arithmetic instructions}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]}
- G06F 15/8046: {Systolic arrays}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/048: Activation functions
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An apparatus includes a global memory and a systolic array. The global memory is configured to store and provide an input feature map (IFM) vector stream from an IFM tensor and a kernel vector stream from a kernel tensor. The systolic array is configured to receive the IFM vector stream and the kernel vector stream from the global memory. The systolic array is on-chip together with the global memory. The systolic array includes a plurality of processing elements (PEs) each having a plurality of vector units, each of the plurality of vector units being configured to perform a dot-product operation on at least one IFM vector of the IFM vector stream and at least one kernel vector of the kernel vector stream per unit clock cycle to generate a plurality of output feature maps (OFMs).

## Claims

### Claim 1

An apparatus, the apparatus comprising:
a global memory configured to store and provide an input feature map (IFM) vector stream from an IFM tensor and a kernel vector stream from a kernel tensor; and
a systolic array configured to receive the IFM vector stream and the kernel vector stream from the global memory, the systolic array being on-chip together with the global memory, the systolic array comprising a plurality of processing elements (PEs) each having a plurality of vector units, each of the plurality of vector units being configured to perform a dot-product operation on at least one IFM vector of the IFM vector stream and at least one kernel vector of the kernel vector stream per unit clock cycle to generate a plurality of output feature maps (OFMs).

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein
the global memory is connected to an IFM fetcher unit, and
the IFM fetcher unit is configured to fetch IFM vectors from the global memory and form the IFM vector stream fed to the systolic array.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein
the IFM fetcher unit comprises buffers respectively dedicated to IFM vector streams, and
the buffers are configured to store the IFM vector streams fed to the systolic array.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein
the global memory is connected to a kernel fetcher unit, and
the kernel fetcher unit is configured to fetch kernel vectors from the global memory and form the kernel vector stream fed to the systolic array.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein a number of kernel vectors fetched by the kernel fetcher unit is equal to a number of vector units available in each of the PEs.

### Claim 6 (depends on 4)

The apparatus of claim 4, wherein
the kernel fetcher unit comprises buffers respectively dedicated to kernel vector streams, and
the buffers are configured to store the kernel vector streams fed to the systolic array.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the IFM vector stream and the kernel vector stream are input to the systolic array, based on identification of an IFM window and a kernel tensor; and streaming of pixels of the IFM window and the kernel tensor, such that relative positions of IFM vectors and kernel vectors input to at least one of the plurality of PEs match.

### Claim 8 (depends on 1)

The apparatus of claim 1, further comprising an OFM write-back unit configured to collect OFM pixels generated from the systolic array and writing the OFM pixels to the global memory.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the plurality of PEs are arranged in an m x n matrix form, wherein m denotes the number of rows, n denotes the number of columns, and m and n are equal.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein
at least one PE from the plurality of PEs in each row receives IFM vectors and transfers the IFM vectors to PEs next to the at least one PE in the same row as the at least one PE, and
PEs in a same row of the systolic array share the same IFM vector stream.

### Claim 11 (depends on 9)

The apparatus of claim 9, wherein
at least one PE from the plurality of PEs in each column receives kernel vectors and transfers the kernel vectors to PEs below the at least one PE in a direction in which kernel vectors are transferred in the same column as the at least one PE, and
PEs in a same column of the systolic array share the same kernel vector stream.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the plurality of vector units are configured to perform the dot-product operations in parallel based on lengths of respective dot-products.

### Claim 13 (depends on 1)

The apparatus of claim 1, wherein each of the plurality of vector units comprises a collection of multiplier hardware and an adder tree for generating OFM pixels.

### Claim 14 (depends on 1)

The apparatus of claim 1, wherein
the global memory comprises a plurality of memory banks, and
each of the plurality of memory banks is assigned to a tensor of a predetermined type at the beginning of a systolic operation.

### Claim 15 (depends on 1)

The apparatus of claim 1, wherein the apparatus is configured to accelerate machine learning operations.

### Claim 16 (depends on 1)

The apparatus of claim 1, wherein the apparatus is a smartphone, a laptop, a desktop, a smart watch, or a smart TV.

### Claim 17

An apparatus, the apparatus comprising:
a global memory configured to store input feature map (IFM) data, weights, kernel data, and output feature map (OFM) data; and
a convolution operation data path engine, connected to the global memory, comprising:
an IFM fetcher unit, connected to the global memory, configured to fetch the IFM data from the global memory and form an IFM vector stream;
a kernel fetcher unit, connected to the global memory, configured to fetch kernel data from the global memory and form a kernel vector stream;
a systolic array, configured to receive the IFM vector stream and the kernel vector stream, comprising a plurality of processing elements (PEs) each having a plurality of vector units, each of the plurality of vector units being configured to perform a dot-product operation on an IFM vector of the IFM vector stream and a kernel vector of the kernel vector stream per unit clock cycle to generate output feature maps (OFMs); and
an OFM write-back unit configured to collect and write the OFMs to the global memory.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the kernel fetcher unit is further configured to fetch kernel vectors from the global memory and form the kernel vector stream.

### Claim 19 (depends on 18)

The apparatus of claim 18, wherein a number of the kernel vectors is equal to a number of vector units available in each of the PEs.

### Claim 20 (depends on 18)

The apparatus of claim 18, wherein
the kernel fetcher unit comprises buffers respectively dedicated to kernel vector streams, and
the buffers are configured to store the kernel vector streams fed to the systolic array.