# US 2020/0401440 A1 · Systems, methods, and apparatuses for heterogeneous computing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401440 A1
- Document ID: US20200401440A1
- Status: Published application
- Kind: A1
- Application number: 16/913,265
- Filed: 2020-06-26
- Published: 2020-12-24
- Earliest claimed: 2019-06-28
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Rajesh M. SANKARAN; Gilbert NEIGER; Narayan RANGANATHAN; Stephen R. VAN DOREN; Joseph NUZMAN; Niall D. MCDONNELL; Michael A. O'HANLON; Lokpraveen B. MOSUR; Tracy Garrett DRYSDALE; Eriko NURVITADHI; Asit K. MISHRA; Ganesh VENKATESH; Deborah T. MARR; Nicholas P. CARTER; Jonathan D. PEARCE; Edward T. GROCHOWSKI; Richard J. GRECO; Robert VALENTINE; Jesus CORBAL; Thomas D. FLETCHER; Dennis R. BRADFORD; Dwight P. MANLEY; Mark J. CHARNEY; Jeffrey J. COOK; Paul CAPRIOLI; Koichi YAMADA; Kent D. GLOSSOP; David B. SHEFFIELD
- Web page: https://patentgrep.com/patent/US20200401440A1

## CPC

- G06F 9/48: Program initiating; Program switching, e.g. by interrupt
- G06F 9/3001: {Arithmetic instructions}
- G06F 9/30014: {with variable precision}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 9/30038: {using a mask}
- G06F 9/3004: {to perform operations on memory}
- G06F 9/30047: {Prefetch instructions; cache control instructions}
- G06F 9/30076: {to perform miscellaneous control operations, e.g. NOP}
- G06F 9/30087: {Synchronisation or serialisation instructions}
- G06F 9/3009: {Thread control instructions}
- G06F 9/30181: {Instruction operation extension or modification}
- G06F 9/30189: {according to execution mode, e.g. mode flag}
- G06F 9/383: {Operand prefetching (cache prefetching G06F12/0862)}
- G06F 9/3834: {Maintaining memory consistency}
- G06F 9/3836: {Instruction issuing, e.g. dynamic instruction scheduling or out of order instruction execution}
- G06F 9/3842: {Speculative instruction execution}
- G06F 9/3851: {from multiple instruction streams, e.g. multistreaming}
- G06F 9/3854: {Instruction completion, e.g. retiring, committing or graduating}
- G06F 9/3858: {Result writeback, i.e. updating the architectural state or memory}
- G06F 9/38585: {with result invalidation, e.g. nullification}
- G06F 9/3863: {using multiple copies of the architectural state, e.g. shadow registers}
- G06F 9/3877: {using a secondary processor, e.g. coprocessor (peripheral processor G06F13/12)}
- G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]}
- G06F 9/3888: {controlled by a single instruction for multiple threads [SIMT] in parallel}
- G06F 9/4411: {Configuring for operating with peripheral devices; Loading of device drivers}
- G06F 9/45504: {Abstract machines for program code execution, e.g. Java virtual machine [JVM], interpreters or emulators}
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06F 2209/509: Offload
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

Embodiments of systems, methods, and apparatuses for heterogeneous computing are described. In some embodiments, a hardware heterogeneous scheduler dispatches instructions for execution on one or more plurality of heterogeneous processing elements, the instructions corresponding to a code fragment to be processed by the one or more of the plurality of heterogeneous processing elements, wherein the instructions are native instructions to at least one of the one or more of the plurality of heterogeneous processing elements.

## Claims

### Claim 1

An apparatus comprising:
a multi-chip package comprising:
an interposer substrate;
a graphics processor die coupled to the interposer substrate, the graphics processor die comprising:
a plurality of data parallel processing circuits to simultaneously perform operations on a plurality of data elements, at least one data parallel processing circuit comprising:
local operand storage to store a plurality of source matrix data elements and a plurality of result matrix data elements of one or more source matrices and result matrices, respectively;
execution circuitry comprising a plurality of dot-product execution circuits to execute a plurality of dot-product instructions in parallel to perform a corresponding plurality of dot product operations on at least a portion of the plurality of source matrix data elements to generate the plurality of destination matrix data elements;
a first multi-protocol on-chip communication fabric coupled to the data parallel processing circuits;
a memory controller coupled to the first multi-protocol on-chip communication fabric;
a plurality of memory dies stacked vertically on the interposer substrate; and
a plurality of memory channels integrated through the interposer substrate to couple the memory controller to the memory dies.

### Claim 2 (depends on 1)

The apparatus of claim 1 wherein the plurality of source matrix data elements comprise floating point data elements.

### Claim 3 (depends on 1)

The apparatus of claim 1 wherein the one or more source matrices comprise a first matrix and wherein a first dot product operation of the simultaneous dot-product operations comprises a dot-product of one or more source matrix data elements from the first matrix and one or more source vector data elements.

### Claim 4 (depends on 3)

The apparatus of claim 3 wherein the first matrix comprises a sparse matrix and wherein the first dot-product operation is to enable a Row-Oriented Sparse Matrix Dense Vector (spMdV) multiplication operation.

### Claim 5 (depends on 3)

The apparatus of claim 3 wherein the first matrix comprises a sparse matrix and wherein the first dot-product operation is to enable a Scale and Update operation.

### Claim 6 (depends on 1)

The apparatus of claim 1further comprising:
virtualization circuitry to map the plurality of data parallel processing circuits to virtual functions and to assign one or more of the virtual functions to one or more virtual machines.

### Claim 7 (depends on 6)

The apparatus of claim 6 wherein the virtualization circuitry comprises one or more control registers to store an indication of the mapping between the virtual functions and the plurality of parallel data processing circuits.

### Claim 8 (depends on 7)

The apparatus of claim 7 wherein the first multi-protocol on-chip communication fabric comprises an arbiter to implement Quality of Service (QoS) and/or Virtual Channels for communication over the first multi-protocol on-chip communication fabric.

### Claim 9 (depends on 1)

The apparatus of claim 1 wherein each memory channel is to interconnect the memory controller to one memory die of the plurality of memory dies.

### Claim 10 (depends on 1)

The apparatus of claim 1 further comprising:
an interconnect to couple the memory controller to a system memory device, wherein the system memory device is coupled to a host processor.

### Claim 11 (depends on 10)

The apparatus of claim 10 further comprising:
memory management circuitry to map a shared virtual memory (SVM) space across the system memory device and the memory dies, the SVM space to be shared by the host processor and the graphics processor die, allowing the host processor and graphics processor die to access the system memory die and the memory dies using a consistent set of virtual memory addresses.

### Claim 12 (depends on 11)

The apparatus of claim 11 wherein the memory management circuitry comprises:
an input-output memory management unit (IOMMU) to provide access by the plurality of data parallel processing circuits to page tables of the host processor.

### Claim 13 (depends on 10)

The apparatus of claim 10 wherein the interconnect comprises a Peripheral Component Interconnect Express (PCIe) interconnect.

### Claim 14

A system comprising:
a system memory;
a multi-chip package comprising:
an interposer substrate;
a graphics processor die coupled to the interposer substrate, the graphics processor die comprising:
a plurality of data parallel processing circuits to simultaneously perform operations on a plurality of data elements, at least one data parallel processing circuit comprising:
local operand storage to store a plurality of source matrix data elements and a plurality of result matrix data elements of one or more source matrices and result matrices, respectively;
execution circuitry comprising a plurality of dot-product execution circuits to execute a plurality of dot-product instructions in parallel to perform a corresponding plurality of dot product operations on at least a portion of the plurality of source matrix data elements to generate the plurality of destination matrix data elements;
a first multi-protocol on-chip communication fabric coupled to the data parallel processing circuits;
a memory controller coupled to the first multi-protocol on-chip communication fabric;
a plurality of memory dies stacked vertically on the interposer substrate;
a plurality of memory channels integrated through the interposer substrate to couple the memory controller to the memory dies; and
a Peripheral Component Interconnect Express (PCIe) interface coupled to the first multi-protocol on-chip communication fabric, the PCIe interface to couple the graphics processor die to the system memory device.

### Claim 15 (depends on 14)

The system of claim 14 wherein the plurality of source matrix data elements comprise floating point data elements.

### Claim 16 (depends on 14)

The system of claim 14 wherein the one or more source matrices comprise a first matrix and wherein a first dot product operation of the simultaneous dot-product operations comprises a dot-product of one or more source matrix data elements from the first matrix and one or more source vector data elements.

### Claim 17 (depends on 16)

The system of claim 16 wherein the first matrix comprises a sparse matrix and wherein the first dot-product operation is to enable a Row-Oriented Sparse Matrix Dense Vector (spMdV) multiplication operation.

### Claim 18 (depends on 16)

The system of claim 16 wherein the first matrix comprises a sparse matrix and wherein the first dot-product operation is to enable a Scale and Update operation.

### Claim 19 (depends on 14)

The system of claim 14 further comprising:
virtualization circuitry to map the plurality of data parallel processing circuits to virtual functions and to assign one or more of the virtual functions to one or more virtual machines.

### Claim 20 (depends on 19)

The system of claim 19 wherein the virtualization circuitry comprises one or more control registers to store an indication of the mapping between the virtual functions and the plurality of parallel data processing circuits.

### Claim 21 (depends on 20)

The system of claim 20 wherein the first multi-protocol on-chip communication fabric comprises an arbiter to implement Quality of Service (QoS) and/or Virtual Channels for communication over the first multi-protocol on-chip communication fabric.

### Claim 22 (depends on 14)

The system of claim 14 wherein each memory channel is to interconnect the memory controller to one memory die of the plurality of memory dies.

### Claim 23 (depends on 14)

The system of claim 14 further comprising:
an interconnect to couple the memory controller to a system memory device, wherein the system memory device is coupled to a host processor.

### Claim 24 (depends on 23)

The system of claim 23 further comprising:
memory management circuitry to map a shared virtual memory (SVM) space across the system memory device and the memory dies, the SVM space to be shared by the host processor and the graphics processor die, allowing the host processor and graphics processor die to access the system memory die and the memory dies using a consistent set of virtual memory addresses.

### Claim 25 (depends on 24)

The system of claim 24 wherein the memory management circuitry comprises:
an input-output memory management unit (IOMMU) to provide access by the plurality of data parallel processing circuits to page tables of the host processor.

### Claim 26 (depends on 14)

The system of claim 14 wherein the graphics processor die further comprises:
a display unit for coupling the graphics processor die to one or more external displays.

### Claim 27

A graphics card comprising:
a Peripheral Component Interconnect Express (PCIe) interface adapted to interface with a PCIe slot of a computer system; and
a multi-chip package comprising:
an interposer substrate;
a graphics processor die coupled to the interposer substrate, the graphics processor die comprising:
a plurality of data parallel processing circuits to simultaneously perform operations on a plurality of data elements, at least one data parallel processing circuit comprising:
local operand storage to store a plurality of source matrix data elements and a plurality of result matrix data elements of one or more source matrices and result matrices, respectively;
execution circuitry comprising a plurality of dot-product execution circuits to execute a plurality of dot-product instructions in parallel to perform a corresponding plurality of dot product operations on at least a portion of the plurality of source matrix data elements to generate the plurality of destination matrix data elements;
a first multi-protocol on-chip communication fabric coupled to the data parallel processing circuits;
a memory controller coupled to the first multi-protocol on-chip communication fabric;
a plurality of memory dies stacked vertically on the interposer substrate;
a plurality of memory channels integrated through the interposer substrate to couple the memory controller to the memory dies; and
PCIe circuitry coupled to the first multi-protocol on-chip communication fabric, the PCIe circuitry to couple the graphics processor die to the system memory device via the PCIe interface.