# US 10,769,748 B2 · Programmable coarse grained and sparse matrix compute hardware with advanced scheduling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,769,748 B2
- Document ID: US10769748B2
- Status: Granted
- Kind: B2
- Application number: 16/197,783
- Filed: 2018-11-21
- Published: 2019-05-09
- Granted: 2020-09-08
- Earliest claimed: 2017-04-28
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Eriko Nurvitadhi; Balaji Vembu; Nicolas C. Galoppo Von Borries; Rajkishore Barik; Tsung-Han Lin; Kamal Sinha; Nadathur Rajagopalan Satish; Jeremy Bottleson; Farshad Akhbari; Altug Koker; Narayan Srinivasa; Dukhwan Kim; Sara S. Baghsorkhi; Justin E. Gottschlich; Feng Chen; Elmoustapha Ould-Ahmed-Vall; Kevin Nealis; Xiaoming Chen; Anbang Yao
- Web page: https://patentgrep.com/patent/US10769748B2

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 9/3001: {Arithmetic instructions}
- G06F 9/3017: {Runtime instruction translation, e.g. macros}
- G06F 9/30196: {using decoder, e.g. decoder per instruction set, adaptable or programmable decoders}
- G06F 9/3851: {from multiple instruction streams, e.g. multistreaming}
- G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]}
- G06F 9/3888: {controlled by a single instruction for multiple threads [SIMT] in parallel}
- G06F 9/38885: {Divergence aspects}
- G06F 9/3895: {for complex operations, e.g. multidimensional or interleaved address generators, macros}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/098: Distributed learning, e.g. federated learning
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

One embodiment provides for a compute apparatus to perform machine learning operations, the compute apparatus comprising a decode unit to decode a single instruction into a decoded instruction, the decoded instruction to cause the compute apparatus to perform a complex machine learning compute operation.

## Claims

### Claim 1

A compute apparatus to perform machine learning operations, the compute apparatus comprising:
a decode unit to decode a single instruction into a decoded instruction, the decoded instruction to cause the compute apparatus to perform a complex machine learning compute operation, wherein the complex machine learning compute operation includes multiple pipeline commands;
a scheduler controller to schedule the multiple pipeline commands to one or more of multiple types of compute units, wherein the multiple types of compute units include a general-purpose graphics compute unit and a near-data compute unit; and
a micro-controller to execute firmware instructions, the firmware instructions to enable a parameter analyzer to determine a type of machine learning operations to perform for the single instruction, wherein the micro-controller is further to offload a near-data compute kernel to the near-data compute unit.

### Claim 2 (depends on 1)

The compute apparatus as in claim 1, wherein the complex machine learning compute operation is to perform a convolution for a layer of a convolutional neural network.

### Claim 3 (depends on 2)

The compute apparatus as in claim 2, wherein the convolution includes multiple matrix operations.

### Claim 4 (depends on 1)

The compute apparatus as in claim 1, additionally including a fetch unit to fetch the single instruction.

### Claim 5 (depends on 4)

The compute apparatus as in claim 4, the fetch unit to store the single instruction to a cache memory.

### Claim 6 (depends on 1)

The compute apparatus as in claim 1, wherein the multiple types of compute units include a sparse compute unit.

### Claim 7 (depends on 1)

The compute apparatus as in claim 1, additionally including a machine learning accelerator to determine a set of operations to perform to execute the decoded instruction.

### Claim 8 (depends on 7)

The compute apparatus as in claim 7, the firmware instructions additionally to enable the machine learning accelerator.

### Claim 9

A non-transitory machine-readable medium storing instructions to cause one or more processors to perform operations comprising:
decoding a single instruction into a decoded instruction, the decoded instruction associated with a set of multiple machine learning operations to be performed via a compute pipeline of a general-purpose graphics processing unit;
determining a set of pipeline commands to perform the set of multiple machine learning operations; and
scheduling the set of pipeline commands to the compute pipeline of the general-purpose graphics processing unit, wherein scheduling the set of pipeline commands to the compute pipeline of the general-purpose graphics processing unit includes scheduling the set of pipeline commands to multiple compute pipelines, the multiple compute pipelines including a general-purpose compute pipeline and a near-data compute pipeline, and scheduling the set of pipeline commands include offloading a near-data compute kernel to the near-data compute pipeline.

### Claim 10 (depends on 9)

The non-transitory machine-readable medium as in claim 9, wherein determining the set of pipeline commands to perform the set of multiple machine learning operations includes analyzing parameters associated with the decoded instruction.

### Claim 11 (depends on 9)

The non-transitory machine-readable medium as in claim 9, additionally comprising retiring the decoded instruction in response completion of the set of pipeline commands.

### Claim 12 (depends on 9)

The non-transitory machine-readable medium as in claim 9, wherein the single instruction is to cause the general-purpose graphics processing unit to perform a convolution for a layer of a convolutional neural network, the convolution including multiple matrix operations.

### Claim 13 (depends on 9)

The non-transitory machine-readable medium as in claim 9, wherein scheduling the set of pipeline commands to the compute pipeline of the general-purpose graphics processing unit includes scheduling one or more pipeline commands in the set of pipeline commands to a sparse compute pipeline of the multiple compute pipelines.

### Claim 14

A data processing system comprising:
a general-purpose graphics processing unit including a fetch unit to fetch a single instruction, a decode unit to decode the single instruction into a decoded instruction, a micro-controller to execute firmware instructions to enable a parameter analyzer to determine a type of machine learning operations to perform for the single instruction, and a scheduler controller to schedule multiple matrix operations to one or more of multiple types of compute units, wherein the multiple types of compute units include a general-purpose graphics compute unit and a near-data compute unit, the decoded instruction is to cause the general-purpose graphics processing unit to execute multiple pipeline commands to perform a complex machine learning compute operation, and the micro-controller is to offload a near-data compute kernel to the near-data compute unit; and
a memory coupled to the general-purpose graphics processing unit.

### Claim 15 (depends on 14)

The data processing system as in claim 14, wherein the multiple types of compute units include a sparse compute unit.

### Claim 16 (depends on 14)

The data processing system as in claim 14, the general-purpose graphics processing unit including a machine learning accelerator to determine the multiple pipeline commands to execute to perform the complex machine learning compute operation.

### Claim 17 (depends on 16)

The data processing system as in claim 16, the firmware instructions to enable the machine learning accelerator.