# US 2025/0061534 A1 · Programmable coarse grained and sparse matrix compute hardware with advanced scheduling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0061534 A1
- Document ID: US20250061534A1
- Status: Published application
- Kind: A1
- Application number: 18/819,073
- Filed: 2024-08-29
- Published: 2025-02-20
- Earliest claimed: 2017-04-28
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Eriko Nurvitadhi; Balaji Vembu; Nicolas C. Galoppo Von Borries; Rajkishore Barik; Tsung-Han Lin; Kamal Sinha; Nadathur Rajagopalan Satish; Jeremy Bottleson; Farshad Akhbari; Altug Koker; Narayan Srinivasa; Dukhwan Kim; Sara S. Baghsorkhi; Justin E. Gottschlich; Feng Chen; Elmoustapha Ould-Ahmed-Vall; Kevin Nealis; Xiaoming Chen; Anbang Yao
- Web page: https://patentgrep.com/patent/US20250061534A1

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 9/3001: {Arithmetic instructions}
- G06F 9/3017: {Runtime instruction translation, e.g. macros}
- G06F 9/30196: {using decoder, e.g. decoder per instruction set, adaptable or programmable decoders}
- G06F 9/3851: {from multiple instruction streams, e.g. multistreaming}
- G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]}
- G06F 9/3888: {controlled by a single instruction for multiple threads [SIMT] in parallel}
- G06F 9/38885: {Divergence aspects}
- G06F 9/3895: {for complex operations, e.g. multidimensional or interleaved address generators, macros}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/098: Distributed learning, e.g. federated learning
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

One embodiment provides a parallel processor comprising a hardware scheduler to schedule pipeline commands for compute operations to one or more of multiple types of compute units, a plurality of processing resources including a first sparse compute unit configured for input at a first level of sparsity and hybrid memory circuitry including a memory controller, a memory interface, and a second sparse compute unit configured for input at a second level of sparsity that is greater than the first level of sparsity.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

An accelerator comprising:
first circuitry to fetch and decode a single instruction into a decoded instruction, the decoded instruction associated with multiple matrix multiply operations to be performed via a compute pipeline of a general-purpose graphics processing unit;
second circuitry to determine a set of pipeline commands to perform the multiple matrix multiply operations;
third circuitry to schedule the set of pipeline commands to the compute pipeline of the general-purpose graphics processing unit; and
fourth circuitry to retire the decoded instruction in response completion of the set of pipeline commands.

### Claim 22 (depends on 21)

The accelerator of claim 21, second circuitry to analyze parameters associated with the decoded instruction to determine the set of pipeline commands to perform the multiple matrix multiply operations.

### Claim 23 (depends on 21)

The accelerator of claim 21, wherein the single instruction is to cause the general-purpose graphics processing unit to perform a convolution for a layer of a convolutional neural network.

### Claim 24 (depends on 21)

The accelerator of claim 21, the third circuitry to schedule the set of pipeline commands to one or more of multiple compute pipelines.

### Claim 25 (depends on 24)

The accelerator of claim 24, the multiple compute pipelines include a general-purpose compute pipeline, a plurality of sparse compute pipelines, and/or a near-data compute pipeline.

### Claim 26 (depends on 25)

The accelerator of claim 25, third circuitry to schedule the set of pipeline commands to one or more of the multiple compute pipelines based on a characteristic of one or more of the multiple matrix multiply operations and/or input data associated with one or more of the multiple matrix multiply operations.

### Claim 27 (depends on 26)

The accelerator of claim 26, third circuitry to schedule the set of pipeline commands to one of the general-purpose compute pipeline and/or one of the plurality of sparse compute pipelines based on a presence of a sparse matrix operation within the set of pipeline commands.

### Claim 28 (depends on 27)

The accelerator of claim 27, third circuitry to schedule the set of pipeline commands to one of the plurality of sparse compute pipelines based on a sparsity characteristic of the input data associated with one or more of the multiple matrix multiply operations, wherein the plurality of sparse compute pipelines include a first sparse compute unit configured for input at a first level of sparsity and a second sparse compute unit configured for input at a second level of sparsity that is greater than the first level of sparsity.

### Claim 29 (depends on 27)

The accelerator of claim 27, third circuitry to schedule the set of pipeline commands to one of the general-purpose compute pipeline, one of the plurality of sparse compute pipelines, and/or the near-data compute pipeline based on a memory access complexity of a workload to be performed by the set of pipeline commands.

### Claim 30

A method comprising:
fetching and decoding a single instruction into a decoded instruction, the decoded instruction associated with multiple matrix multiply operations to be performed via a compute pipeline of a general-purpose graphics processing unit;
determining a set of pipeline commands to perform the multiple matrix multiply operations;
scheduling the set of pipeline commands to the compute pipeline of the general-purpose graphics processing unit; and
retiring the decoded instruction in response completion of the set of pipeline commands.

### Claim 31 (depends on 30)

The method of claim 30, comprising analyzing parameters associated with the decoded instruction to determine the set of pipeline commands to perform the multiple matrix multiply operations.

### Claim 32 (depends on 30)

The method of claim 30, wherein the single instruction is to cause the general-purpose graphics processing unit to perform a convolution for a layer of a convolutional neural network.

### Claim 33 (depends on 30)

The method of claim 30, wherein scheduling the set of pipeline commands to the compute pipeline of the general-purpose graphics processing unit includes scheduling the set of pipeline commands to one or more of multiple compute pipelines.

### Claim 34 (depends on 33)

The method of claim 33, the multiple compute pipelines include a general-purpose compute pipeline, a plurality of sparse compute pipelines, and/or a near-data compute pipeline.

### Claim 35 (depends on 34)

The method of claim 34, comprising scheduling the set of pipeline commands to one or more of the multiple compute pipelines based on a characteristic of one or more of the multiple matrix multiply operations and/or input data associated with one or more of the multiple matrix multiply operations.

### Claim 36 (depends on 35)

The method of claim 35, comprising scheduling the set of pipeline commands to one of the general-purpose compute pipeline and/or one of the plurality of sparse compute pipelines based on a presence of a sparse matrix operation within the set of pipeline commands.

### Claim 37 (depends on 36)

The method of claim 36, comprising scheduling the set of pipeline commands to one of the plurality of sparse compute pipelines based on a sparsity characteristic of the input data associated with one or more of the multiple matrix multiply operations, wherein the plurality of sparse compute pipelines include a first sparse compute unit configured for input at a first level of sparsity and a second sparse compute unit configured for input at a second level of sparsity that is greater than the first level of sparsity.

### Claim 38 (depends on 36)

The method of claim 36, comprising scheduling the set of pipeline commands to one of the general-purpose compute pipeline, one of the plurality of sparse compute pipelines, and/or the near-data compute pipeline based on a memory access complexity of a workload to be performed by the set of pipeline commands.

### Claim 39

A data processing system comprising:
a memory device; and
an accelerator coupled with the memory device, the accelerator comprising:
first circuitry to fetch and decode a single instruction into a decoded instruction, the decoded instruction associated with multiple matrix multiply operations to be performed via a compute pipeline of a general-purpose graphics processing unit;
second circuitry to determine a set of pipeline commands to perform the multiple matrix multiply operations;
third circuitry to schedule the set of pipeline commands to the compute pipeline of the general-purpose graphics processing unit; and
fourth circuitry to retire the decoded instruction in response completion of the set of pipeline commands.

### Claim 40 (depends on 39)

The data processing system of claim 39, wherein:
the second circuitry is to analyze parameters associated with the decoded instruction to determine the set of pipeline commands to perform the multiple matrix multiply operations, wherein the single instruction is to cause the general-purpose graphics processing unit to perform a convolution for a layer of a convolutional neural network; and
the third circuitry is to schedule the set of pipeline commands to one or more of multiple compute pipelines, the multiple compute pipelines including a general-purpose compute pipeline, a plurality of sparse compute pipelines, and a near-data compute pipeline.
