# US 2018/0322390 A1 · Optimized compute hardware for machine learning operations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322390 A1
- Document ID: US20180322390A1
- Status: Published application
- Kind: A1
- Application number: 15/869,564
- Filed: 2018-01-12
- Published: 2018-11-08
- Earliest claimed: 2017-05-05
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Dipankar Das; Roger Gramunt; Mikhail Smelyanskiy; Jesus Corbal; Dheevatsa Mudigere; Naveen K. Mellempudi; Alexander F. Heinecke
- Web page: https://patentgrep.com/patent/US20180322390A1

## CPC

- G06F 9/3888: {controlled by a single instruction for multiple threads [SIMT] in parallel}
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 9/30014: {with variable precision}
- G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations}
- G06F 9/3016: {Decoding the operand specifier, e.g. specifier format}
- G06F 9/30181: {Instruction operation extension or modification}
- G06F 9/30192: {according to data descriptor, e.g. dynamic data typing}
- G06F 9/3851: {from multiple instruction streams, e.g. multistreaming}
- G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06F 2207/382: Reconfigurable for different fixed word lengths
- G06N 3/048: Activation functions
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

One embodiment provides for a compute apparatus to perform machine learning operations, the compute apparatus comprising a fetch unit to fetch a single instruction having multiple input operands, wherein the multiple input operands have an unequal bit-length, a first input operand having a first bit-length and a second input operand having a second bit-length; a decode unit to decode the single instruction into a decoded instruction; an operand length unit to determine a smaller bit-length of the first bit-length and the second bit-length; and a compute unit to perform a matrix operation on the multiple input operands to generate an output value having a bit length of the smaller bit length.

## Claims

### Claim 1

A compute apparatus to perform machine learning operations, the compute apparatus comprising:
a fetch unit to fetch a single instruction having multiple input operands, wherein the multiple input operands have an unequal bit-length, a first input operand having a first bit-length and a second input operand having a second bit-length;
a decode unit to decode the single instruction into a decoded instruction;
an operand length unit to determine a smaller bit-length of the first bit-length and the second bit-length; and
a compute unit to perform a matrix operation on the multiple input operands to generate an output value having a bit length of the smaller bit length.

### Claim 2 (depends on 1)

The compute apparatus as in claim 1, wherein the single instruction is to cause the compute unit to perform an at least two-dimensional matrix multiply operation on the multiple operands.

### Claim 3 (depends on 2)

The compute apparatus as in claim 2, wherein the single instruction is to cause the compute unit to perform at least one multiply-add operation.

### Claim 4 (depends on 2)

The compute apparatus as in claim 2, wherein the single instruction is to cause the compute unit to perform at least one multiply-accumulate operation.

### Claim 5 (depends on 2)

The compute apparatus as in claim 2, wherein the single instruction is to cause the compute unit to perform an at least two-dimensional matrix dot product operation.

### Claim 6 (depends on 1)

The compute apparatus as in claim 1, additionally including a precision configuration unit to provide a precision configuration to the compute unit.

### Claim 7 (depends on 6)

The compute apparatus as in claim 6, the precision configuration unit to configure an operational precision of a functional unit within the compute unit.

### Claim 8 (depends on 1)

The compute apparatus as in claim 1, the compute unit to a perform a matrix multiply operation on the multiple operands, wherein the compute unit is to:
determine whether a multiply operation of the matrix multiply operation has a zero input; and
bypass the multiply operation having the zero input.

### Claim 9 (depends on 8)

The compute apparatus as in claim 8, wherein to bypass the multiply operation having the zero input, the compute unit is to bypass scheduling of the multiply operation having the zero input.

### Claim 10 (depends on 8)

The compute apparatus as in claim 8, wherein to bypass the multiply operation having the zero input, the compute unit is to:
determine whether a multiply operation of the matrix multiply operation has a zero input: and
substitute a no-operation opcode for a multiply operation opcode for a functional unit associated with the zero input.

### Claim 11

A method of executing a number of passes to execute a multi-pass matrix multiply operation on a general-purpose graphics processing unit (GPGPU), the method comprising:
fetching, within the GPGPU, a single instruction to be executed;
decoding the instruction to determine a type for the instruction;
in response to determining the type for the instruction, determining the instruction is a multi-pass matrix instruction including multiply operations, and determining whether a threshold number of multiply operations have a zero-value input operand;
compacting the multiply operations without a zero-value input operand into a reduced number of passes; and
performing the instruction in a reduced number of passes.

### Claim 12 (depends on 11)

The method as in claim 11, wherein performing the instruction in the reduced number of passes includes performing a two-pass matrix multiply instruction in a single pass when at least half of the multiply operations have one or more zero-value input operands.

### Claim 13 (depends on 11)

The method as in claim 11, wherein determining whether the threshold number of multiply operations have a zero-value input operand includes:
comparing input operands for a multiply operation to zero; and
tagging the multiply operation as having a zero output.

### Claim 14 (depends on 13)

The method as in claim 13, wherein tagging the multiply operations as having a zero output includes applying a tag to a destination register associated with the multiply operation.

### Claim 15 (depends on 13)

The method as in claim 13, wherein compacting the multiply operations without a zero-value input operand into a reduced number of passes includes:
scheduling multiply operations with non-zero values on a compute unit of the GPGPU;
bypassing scheduling of multiply operations having a zero output.

### Claim 16 (depends on 13)

The method as in claim 13, wherein the instruction is a multi-pass matrix instruction including multiply-add operations and performing the instruction in a reduced number of passes includes writing an input operand for an add operation to a destination register.

### Claim 17

A data processing system comprising:
a non-transitory machine-readable medium to store instructions for execution by one or more processors of the data processing system; and
a general-purpose graphics processing unit including instruction decode logic to decode a single instruction including multiple operands into a single decoded instruction, the multiple operands including multiple input operands having an unequal bit-length, the multiple input operands including a first input operand having a first bit-length and a second input operand having a second bit-length, wherein the general-purpose graphics processing unit is to generate an output value having a bit length of the smaller bit length.

### Claim 18 (depends on 17)

The data processing system as in claim 17, the general-purpose graphics processing unit including:
an operand length unit to determine a smaller bit-length of the first bit-length and the second bit-length; and
a compute unit to perform a matrix operation on the multiple operands to generate the output value having a bit-length equal to the smaller bit-length.

### Claim 19 (depends on 18)

The data processing system as in claim 18, wherein the single instruction is to cause the compute unit to perform an at least two-dimensional matrix multiply operation on the multiple operands.

### Claim 20 (depends on 18)

The data processing system as in claim 18, wherein the single instruction is to cause the compute unit to perform at least one multiply-add operation, an at least two-dimensional matrix dot product operation, or at least one multiply-accumulate operation.
