# US 11,030,095 B2 · Virtual space memory bandwidth reduction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,030,095 B2
- Document ID: US11030095B2
- Status: Granted
- Kind: B2
- Application number: 16/215,298
- Filed: 2018-12-10
- Published: 2020-06-11
- Granted: 2021-06-08
- Earliest claimed: 2018-12-10
- Assignees: Advanced Micro Devices, Inc.
- Applicants: Advanced Micro Devices, Inc.
- Inventors: Swapnil Sakharshete; Samuel Lawrence Wasmundt
- Web page: https://patentgrep.com/patent/US11030095B2

## CPC

- G06F 12/0646: {Configuration or reconfiguration}
- G06T 1/60: Memory management
- G06F 12/0207: {with multidimensional access, e.g. row/column, matrix}
- G06F 12/0811: with multilevel cache hierarchies
- G06F 12/109: for multiple virtual address spaces, e.g. segmentation (G06F12/1036 takes precedence)
- G06F 17/153: {Multidimensional correlation or convolution}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 2212/1041: Resource optimization
- G06F 2212/1044: Space efficiency improvement
- G06F 2212/454: Vector or matrix data
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A processing system includes a central processing unit (CPU) and a graphics processing unit (GPU) that has a plurality of compute units. The GPU receives an image from the CPU and determines a total result area in a virtual-matrix-multiplication space of a virtual matrix-multiplication output matrix based on convolutional parameters associated with the image in an image space. The GPU partitions the total result area of the virtual matrix-multiplication output matrix into a plurality of virtual segments. The GPU allocates convolution operations to the plurality of compute units based on each virtual segment of the plurality of virtual segments.

## Claims

### Claim 1

A method, comprising:
ascertaining convolutional parameters associated with an image in an image space;
determining, based on said convolutional parameters, a total result area in a virtual matrix-multiplication space of a virtual matrix-multiplication output matrix;
partitioning said total result area of said virtual matrix-multiplication output matrix into a plurality of virtual segments; and
allocating convolution operations to a plurality of compute units based on each virtual segment of said plurality of virtual segments.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating a matrix-multiplication space filter matrix based on filter dimensions of an image space filter; and
generating a virtual matrix-multiplication space matrix based on said filter dimensions and said convolutional parameters.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
generating a matrix-multiplication space filter matrix segment of said matrix-multiplication space filter matrix and a virtual matrix-multiplication space matrix segment of said virtual matrix-multiplication space matrix based on dimensions of each virtual segment of said plurality of virtual segments.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
performing a first convolution operation of said convolution operations based on said matrix-multiplication space filter matrix segment and said virtual matrix-multiplication space matrix segment.

### Claim 5 (depends on 3)

The method of claim 3, further comprising:
mapping each point XY of said virtual matrix-multiplication space matrix segment to a pixel in said image of said image space.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
fetching virtual matrix-multiplication space matrix data from said image by linearizing a pixel coordinate of said pixel to create an address into a memory where said image is stored.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
fetching matrix-multiplication space filter matrix data from said memory.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
performing matrix multiplication between said virtual matrix-multiplication space matrix data and matrix-multiplication space filter matrix data to generate a convolution result for virtual matrix-multiplication output matrix.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
transforming said convolutional result of said virtual matrix-multiplication output matrix to image space form.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
determining whether there is duplicate data in convolutional data associated with said convolution operations by performing a mapping operation.

### Claim 11 (depends on 10)

The method of claim 10, wherein:
said virtual matrix-multiplication space is a virtual GEMM space and said virtual matrix-multiplication output matrix is a virtual GEMM output matrix.

### Claim 12

A processing system, comprising:
a processor configured to be coupleable to a memory, said processor having a plurality of compute units, wherein said processor is configured to receive an image from said memory, determine a total result area in a virtual matrix-multiplication space of a virtual matrix-multiplication output matrix based on convolutional parameters associated with said image in an image space, partition said total result area of said virtual matrix-multiplication output matrix into a plurality of virtual segments, and allocate convolution operations to a plurality of compute units based on each virtual segment of said plurality of virtual segments.

### Claim 13 (depends on 12)

The processing system of claim 12, wherein said processor is configured to generate a virtual matrix-multiplication space virtual segment and a virtual matrix-multiplication space filter virtual segment based on dimensions of a virtual segment of said plurality of virtual segments.

### Claim 14 (depends on 13)

The processing system of claim 13, wherein said processor is configured to perform a first convolution operation of said convolution operations based on said virtual matrix-multiplication space matrix segment and said matrix-multiplication space filter matrix segment.

### Claim 15 (depends on 12)

The processing system of claim 12, wherein said processor is configured to determine whether there is duplicate data in convolutional data associated with said convolution operations.

### Claim 16 (depends on 12)

The processing system of claim 12, further comprising:
an internal cache hierarchy and a global memory, wherein said processor is configured to place duplicate data in said internal cache hierarchy and accesses said duplicate data from said internal cache hierarchy instead of said global memory.

### Claim 17

A method comprising:
receiving image data into a global memory of a processor;
placing said image data into an internal cache hierarchy of said processor;
determining which data associated with convolutional data of said image data is duplicate data; and
accessing said duplicate data from said internal cache hierarchy based on said determination of whether said data is said duplicate data;
generating a virtual GEMM output matrix based on convolutional parameters associated with said image data; and
partitioning a total result area of a virtual GEMM space output matrix into a plurality of virtual segments, wherein determining which data associated with said convolutional data is said duplicate data is based on an address assessment of duplicative memory requests by said processor.

### Claim 18 (depends on 17)

The method of claim 17, allocating a convolution operation to a first compute unit of a plurality of compute units based on said convolutional parameters associated with said image data.