# US 2020/0394468 A1 · Tiling format for convolutional neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394468 A1
- Document ID: US20200394468A1
- Status: Published application
- Kind: A1
- Application number: 17/006,533
- Filed: 2020-08-28
- Published: 2020-12-17
- Earliest claimed: 2018-12-20
- Assignees: Advanced Micro Devices, Inc.
- Applicants: Advanced Micro Devices, Inc.
- Inventors: Song Zhang; Jiantan Liu; Hua Zhang; Min Yu
- Web page: https://patentgrep.com/patent/US20200394468A1

## CPC

- G06N 3/08: Learning methods
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06V 10/449: {Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters}
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 10/82: using neural networks
- G06V 10/955: {using specific electronic processors}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Systems, apparatuses, and methods for converting data to a tiling format when implementing convolutional neural networks are disclosed. A system includes at least a memory, a cache, a processor, and a plurality of compute units. The memory stores a first buffer and a second buffer in a linear format, where the first buffer stores convolutional filter data and the second buffer stores image data. The processor converts the first and second buffers from the linear format to third and fourth buffers, respectively, in a tiling format. The plurality of compute units load the tiling-formatted data from the third and fourth buffers in memory to the cache and then perform a convolutional filter operation on the tiling-formatted data. The system generates a classification of a first dataset based on a result of the convolutional filter operation.

## Claims

### Claim 1

A system comprising:
a memory storing a first buffer and a second buffer in a linear format, wherein the first buffer stores convolutional filter data and the second buffer stores image data;
a cache;
a processor configured to:
convert the convolutional filter data in the first buffer from the linear format to a tiling format and store in a third buffer; and
convert the image data in the second buffer from the linear format to the tiling format and store in a fourth buffer; and
a plurality of compute units, wherein the plurality of compute units are configured to:
load data elements of the third buffer and the fourth buffer from the memory to the cache; and
perform a convolutional filter operation on the data elements of the third buffer and the fourth buffer to generate a result in the tiling format;
wherein the system is configured to generate a classification of a first dataset based on the result from the convolutional filter operation.

### Claim 2 (depends on 1)

The system as recited in claim 1, wherein the cache comprises a plurality of channels, wherein the plurality of compute units are configured to load convolutional filter data elements from the third buffer on the plurality of channels in parallel in a single clock cycle.

### Claim 3 (depends on 1)

The system as recited in claim 1, wherein the cache comprises a plurality of channels, wherein the plurality of compute units are configured to load image data elements from the fourth buffer on the plurality of channels in parallel in a single clock cycle.

### Claim 4 (depends on 1)

The system as recited in claim 1, wherein converting the first buffer from the linear format to the third buffer in the tiling format comprises:
reading values from sequential locations of the first buffer in the memory; and
stepping through the third buffer with a stride equal to a sum of a first number of input channels and a second number of convolutional filters while writing the values to the third buffer.

### Claim 5 (depends on 1)

The system as recited in claim 1, wherein converting the second buffer from the linear format to the fourth buffer in the tiling format comprises:
reading values from sequential locations of the second buffer in the memory; and
stepping through the fourth buffer with a stride equal to a number of pixel channels while writing the values to the fourth buffer.

### Claim 6 (depends on 1)

The system as recited in claim 1, wherein the convolutional filter data comprises a plurality of convolutional filters, wherein each convolutional filter of the plurality of convolutional filters has three rows and three columns.

### Claim 7 (depends on 1)

The system as recited in claim 1, wherein the classification of the first dataset is generated during execution of a machine learning engine application.

### Claim 8

A method comprising:
storing, by a memory, a first buffer and a second buffer in a linear format, wherein the first buffer stores convolutional filter data and the second buffer stores image data;
converting, by a processor, the convolutional filter data in the first buffer from the linear format to a tiling format and store in a third buffer;
converting, by a processor, the image data in the second buffer from the linear format to the tiling format and store in a fourth buffer;
loading, by a plurality of compute units, data elements of the third buffer and the fourth buffer from the memory to a cache;
performing, by a plurality of compute units, a convolutional filter operation on the data elements of the third buffer and the fourth buffer to generate a result in the tiling format; and
generating a classification of a first dataset based on the result from the convolutional filter operation.

### Claim 9 (depends on 8)

The method as recited in claim 8, wherein the cache comprises a plurality of channels, and wherein the method further comprising loading, by the plurality of compute units, convolutional filter data from the third buffer on the plurality of channels in parallel in a single clock cycle.

### Claim 10 (depends on 8)

The method as recited in claim 8, wherein the cache comprises a plurality of channels, and wherein the method further comprising loading, by the plurality of compute units, image data from the fourth buffer on the plurality of channels in parallel in a single clock cycle.

### Claim 11 (depends on 8)

The method as recited in claim 8, wherein converting the first buffer from the linear format to the third buffer in the tiling format comprises:
reading values from sequential locations of the first buffer in the memory; and
stepping through the third buffer with a stride equal to a sum of a first number of input channels and a second number of convolutional filters while writing the values to the third buffer.

### Claim 12 (depends on 8)

The method as recited in claim 8, wherein converting the second buffer from the linear format to the fourth buffer in the tiling format comprises:
reading values from sequential locations of the second buffer in the memory; and
stepping through the fourth buffer with a stride equal to a number of pixel channels while writing the values to the fourth buffer.

### Claim 13 (depends on 8)

The method as recited in claim 8, wherein the convolutional filter data comprises a plurality of convolutional filters, wherein each convolutional filter of the plurality of convolutional filters has three rows and three columns.

### Claim 14 (depends on 8)

The method as recited in claim 8, further comprising generating the classification of the first dataset during execution of a machine learning engine application.

### Claim 15

An apparatus comprising:
a memory storing a first buffer and a second buffer in a linear format, wherein the first buffer stores convolutional filter data and the second buffer stores image data; and
a processor configured to:
convert the convolutional filter data in the first buffer from the linear format to a tiling format and store in a third buffer; and
convert the image data in the second buffer from the linear format to the tiling format and store in a fourth buffer; and
generate a classification of a first dataset based on a result from a convolutional filter operation performed on data elements stored in the tiling format in the third buffer and the fourth buffer.

### Claim 16 (depends on 15)

The apparatus as recited in claim 15, further comprising a cache, wherein the cache comprises a plurality of channels, and wherein the apparatus is further configured to load convolutional filter data from the third buffer on the plurality of channels in parallel in a single clock cycle.

### Claim 17 (depends on 15)

The apparatus as recited in claim 15, further comprising a cache, wherein the cache comprises a plurality of channels, and wherein the apparatus is further configured to load image data from the fourth buffer on the plurality of channels in parallel in a single clock cycle.

### Claim 18 (depends on 15)

The apparatus as recited in claim 15, wherein converting the first buffer from the linear format to the third buffer in the tiling format comprises:
reading values from sequential locations of the first buffer in the memory; and
stepping through the third buffer with a stride equal to a sum of a first number of input channels and a second number of convolutional filters while writing the values to the third buffer.

### Claim 19 (depends on 15)

The apparatus as recited in claim 15, wherein converting the second buffer from the linear format to the fourth buffer in the tiling format comprises:
reading values from sequential locations of the second buffer in the memory; and
stepping through the fourth buffer with a stride equal to a number of pixel channels while writing the values to the fourth buffer.

### Claim 20 (depends on 15)

The apparatus as recited in claim 15, wherein the classification of the first dataset is generated during execution of a machine learning engine application.
