# US 12,007,935 B2 · Graphics processors and graphics processing units having dot product accumulate instruction for hybrid floating point format

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,007,935 B2
- Document ID: US12007935B2
- Status: Granted
- Kind: B2
- Application number: 17/428,523
- Filed: 2020-03-14
- Published: 2022-04-28
- Granted: 2024-06-11
- Earliest claimed: 2019-03-15
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Subramaniam Maiyuran; Shubra Marwaha; Ashutosh Garg; Supratim Pal; Jorge Parra; Chandra Gurram; Varghese George; Darin Starkey; Guei-Yuan Lueh
- Web page: https://patentgrep.com/patent/US12007935B2

## CPC

- G06F 15/7839: {with memory}
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 7/575: Basic arithmetic logic units, i.e. devices selectable to perform either addition, subtraction or one of several logical operations, using, at least partially, the same circuitry
- G06F 7/58: Random or pseudo-random number generators
- G06F 7/588: {Random number generators, i.e. based on natural stochastic processes}
- G06F 9/3001: {Arithmetic instructions}
- G06F 9/30014: {with variable precision}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 9/3004: {to perform operations on memory}
- G06F 9/30043: {LOAD or STORE instructions; Clear instruction}
- G06F 9/30047: {Prefetch instructions; cache control instructions}
- G06F 9/30065: {Loop control instructions; iterative instructions, e.g. LOOP, REPEAT}
- G06F 9/30079: {Pipeline control instructions, e.g. multicycle NOP}
- G06F 9/383: {Operand prefetching (cache prefetching G06F12/0862)}
- G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]}
- G06F 9/3888: {controlled by a single instruction for multiple threads [SIMT] in parallel}
- G06F 9/5011: {the resources being hardware resources other than CPUs, Servers and Terminals}
- G06F 9/5066: {Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs (mappping at compile time, see G06F8/451)}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 12/0215: {with look ahead addressing means}
- G06F 12/0238: {Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory}
- G06F 12/0246: {in block erasable memory, e.g. flash memory}
- G06F 12/0607: {Interleaved addressing}
- G06F 12/0802: Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
- G06F 12/0804: with main memory updating (G06F12/0806 takes precedence)
- G06F 12/0811: with multilevel cache hierarchies
- G06F 12/0862: with prefetch
- G06F 12/0866: for peripheral storage systems, e.g. disk cache
- G06F 12/0871: Allocation or management of cache space
- G06F 12/0875: with dedicated cache, e.g. instruction or stack
- G06F 12/0882: Page mode
- G06F 12/0888: using selective caching, e.g. bypass
- G06F 12/0891: using clearing, invalidating or resetting means
- G06F 12/0893: Caches characterised by their organisation or structure
- G06F 12/0895: of parts of caches, e.g. directory or tag array
- G06F 12/0897: with two or more cache hierarchy levels (with multilevel cache hierarchies G06F12/0811)
- G06F 12/1009: using page tables, e.g. page table structures
- G06F 12/128: adapted to multidimensional cache systems, e.g. set-associative, multicache, multiset or multilevel
- G06F 13/1626: {by reordering requests}
- G06F 15/173: using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
- G06F 15/8046: {Systolic arrays}
- G06F 16/24569: {Query processing with adaptation to specific hardware, e.g. adapted for using GPUs or SSDs}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 1/60: Memory management
- H03M 7/46: Conversion to or from run-length codes, i.e. by representing the number of consecutive digits, or groups of digits, of the same kind by a code word and a digit indicative of that kind
- G06F 9/3802: {Instruction prefetching}
- G06F 9/3818: {Decoding for concurrent execution}
- G06F 9/3867: {using instruction pipelines}
- G06F 12/12: Replacement control
- G06F 2212/1008: Correctness of operation, e.g. memory ordering
- G06F 2212/1016: Performance improvement
- G06F 2212/1021: Hit rate improvement
- G06F 2212/1024: Latency reduction
- G06F 2212/1044: Space efficiency improvement
- G06F 2212/2542: Non-uniform memory access [NUMA] architecture
- G06F 2212/302: In image processor or graphics adapter
- G06F 2212/401: Compressed data
- G06F 2212/455: Image or video data
- G06F 2212/60: Details of cache memory
- G06F 2212/601: Reconfiguration of cache memory
- G06F 2212/6026: Prefetching based on access pattern detection, e.g. stride based prefetch
- G06F 2212/6028: Prefetching based on hints or prefetch instructions
- G06F 2212/608: Details relating to cache mapping
- G06F 2212/652: Page size control
- G06N 3/08: Learning methods
- G06T 15/06: Ray-tracing
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)

## Abstract

Graphics processors and graphics processing units having dot product accumulate instructions for a hybrid floating point format are disclosed. In one embodiment, a graphics multiprocessor comprises an instruction unit to dispatch instructions and a processing resource coupled to the instruction unit. The processing resource is configured to receive a dot product accumulate instruction from the instruction unit and to process the dot product accumulate instruction using a bfloat16 number (BF16) format.

## Claims

### Claim 1

A graphics multiprocessor, comprising:
an instruction unit to dispatch instructions; and
a processing resource coupled to the instruction unit, the processing resource of the graphics multiprocessor is configured to receive a dot product accumulate instruction from the instruction unit and to process the dot product accumulate instruction using a bfloat16 number (BF16) format.

### Claim 2 (depends on 1)

The graphics multiprocessor of claim 1, wherein the dot product accumulate instruction causes a second source operand to multiply a third source operand while an accumulator adds a first source operand with output from multiplying the second source operand and the third source operand.

### Claim 3 (depends on 2)

The graphics multiprocessor of claim 2, wherein the accumulator generates an output for a destination.

### Claim 4 (depends on 2)

The graphics multiprocessor of claim 2, wherein the first source operand comprises a single-precision floating point format while at least one of the second and third source operands comprise BF16 format.

### Claim 5 (depends on 2)

The graphics multiprocessor of claim 2 wherein the first source operand and the destination are half-precision floating point format, single-precision floating point format, or BF16 formats.

### Claim 6 (depends on 1)

The graphics multiprocessor of claim 1 wherein the processing resource comprises a floating point unit (FPU) to execute the dot product accumulate instruction using the BF16 format.

### Claim 7 (depends on 1)

The graphics multiprocessor of claim 1, wherein the instruction unit to dispatch instructions comprising single instruction multiple data (SIMD) instructions, wherein the processing resource is configured to apply a rectified linear unit function to a result of the add.

### Claim 8

A general-purpose graphics processing unit (GPGPU) core comprising:
a single precision floating-point unit for single precision floating point operations; and
a half-precision floating point unit for half-precision floating point operations, the half-precision floating point unit of the GPGPU core is configured to execute a dot product accumulate instruction using a bfloat16 (BF16) format.

### Claim 9 (depends on 8)

The GPGPU core of claim 8, wherein the dot product accumulate instruction causes first and second multipliers to each multiply second and third source operands while an accumulator adds a first source operand with output from each of the first and second multipliers.

### Claim 10 (depends on 9)

The GPGPU core of claim 9, wherein the accumulator generates an output for a destination.

### Claim 11 (depends on 9)

The GPGPU core of claim 9, wherein the first source operand comprises a single-precision floating point format while at least one of the second and third source operands comprise BF16 format.

### Claim 12 (depends on 9)

The GPGPU core of claim 9 wherein the first source operand and the destination are half-precision floating point format, single-precision floating point format, or BF16 formats.

### Claim 13 (depends on 8)

The GPGPU core of claim 8 wherein the dot product accumulate instruction causes a first stage of first and second BF16 multipliers to each multiply second and third source operands while an accumulator adds a first source operand with output from each of the first and second multipliers to generate an output of the first stage.

### Claim 14 (depends on 13)

The GPGPU core of claim 13, wherein the dot product accumulate instruction for a cascaded arrangement with N stages of multipliers and accumulators causes a second stage of first and second multipliers to each multiply second and third source operands while an accumulator adds the output from the first stage with output from each of the first and second BF16 multipliers of the second stage.

### Claim 15

A parallel processing unit comprising:
a first processing cluster to perform parallel processing operations; and
a second processing cluster coupled to the first processing cluster, wherein the first processing cluster of the parallel processing unit includes a floating-point unit to perform floating point operations, the floating-point unit is configured to process a dot product accumulate instruction using a bfloat16 (BF16) format.

### Claim 16 (depends on 15)

The parallel processing unit of claim 15, wherein the dot product accumulate instruction causes first and second multipliers to each multiply second and third source operands while an accumulator adds a first source operand with output from each of the first and second multipliers.

### Claim 17 (depends on 16)

The parallel processing unit of claim 16, wherein the accumulator generates an output for a destination.

### Claim 18 (depends on 16)

The parallel processing unit of claim 16, wherein the first source operand comprises a single-precision floating point format while at least one of the second and third source operands comprise BF16 format.

### Claim 19 (depends on 16)

The parallel processing unit of claim 16, wherein the first source operand and the destination are half-precision floating point format, single-precision floating point format, or BF16 formats.

### Claim 20 (depends on 15)

The parallel processing unit of claim 15, wherein the dot product accumulate instruction causes a first stage of first and second multipliers to each multiply second and third source operands while an accumulator adds a first source operand with output from each of the first and second multipliers, wherein the dot product accumulate instruction for a cascaded arrangement with N stages of multipliers and accumulators causes a second stage of first and second multipliers to each multiply second and third source operands while an accumulator adds the output from the first stage with output from each of the first and second multipliers of the second stage.

### Claim 21 (depends on 15)

The parallel processing unit of claim 15, wherein the floating-point unit comprises a cascaded arrangement with N stages of multipliers and accumulators.

### Claim 22 (depends on 21)

The parallel processing unit of claim 21, wherein the N stages comprise:
a first stage of first and second multipliers to each multiply second and third source operands and an accumulator to add a first source operand with output from each of the first and second multipliers to generate output of the first stage; and
a second stage of first and second multipliers to each multiply second and third source operands and an accumulator to add the output from the first stage with output from each of the first and second multipliers of the second stage.

### Claim 23

A computing device, comprising:
input/output (I/O) devices;
a central processing unit (CPU) coupled to the I/O devices;
a graphics processing unit (GPU) coupled to the CPU, the GPU having a core that is configured to receive a dot product accumulate instruction and to process the dot product accumulate instruction using a bfloat16 number (BF16) format.

### Claim 24 (depends on 23)

The computing device of claim 23, wherein the dot product accumulate instruction causes first and second multipliers to each multiply second and third source operands while an accumulator adds a first source operand with output from each of the first and second multipliers.

### Claim 25 (depends on 24)

The computing device of claim 24, wherein the accumulator is configured to generate an output for a destination.