# US 2018/0322607 A1 · Dynamic precision management for integer deep learning primitives

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322607 A1
- Document ID: US20180322607A1
- Status: Published application
- Kind: A1
- Application number: 15/881,991
- Filed: 2018-01-29
- Published: 2018-11-08
- Earliest claimed: 2017-05-05
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Naveen MELLEMPUDI; DHEEVATSA MUDIGERE; DIPANKAR DAS; SRINIVAS SRIDHARAN
- Web page: https://patentgrep.com/patent/US20180322607A1

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06F 5/01: for shifting, e.g. justifying, scaling, normalising {(digital stores in which the information is moved stepwise, e.g. shift-registers G11C19/00; digital stores in which the information circulates G11C21/00)}
- G06F 7/501: Half or full adders, i.e. basic adder cells for one denomination
- G06F 7/523: Multiplying only
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06F 17/153: {Multidimensional correlation or convolution}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/098: Distributed learning, e.g. federated learning
- G06F 2207/382: Reconfigurable for different fixed word lengths
- G06F 2207/4824: Neural networks
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

One embodiment provides for a graphics processing unit to perform computations associated with a neural network, the graphics processing unit comprising compute unit including a hardware logic unit having dynamic precision fixed-point logic; a decode unit to decode an instruction for execution by the compute unit, the instruction to cause the compute unit to perform a matrix arithmetic operation on a set of dynamic fixed-point tensors; and a dynamic precision manager to dynamically adjust the precision of a compute operation performed by the compute unit during the matrix arithmetic operation, the dynamic precision manager to adjust the precision of the compute operation to prevent an arithmetic overflow.

## Claims

### Claim 1

A graphics processing unit to perform computations associated with a neural network, the graphics processing unit comprising:
compute unit including a hardware logic unit having dynamic precision fixed-point logic;
a decode unit to decode an instruction for execution by the compute unit, the instruction to cause the compute unit to perform a matrix arithmetic operation on a set of dynamic fixed-point tensors; and
a dynamic precision manager to dynamically adjust the precision of a compute operation performed by the compute unit during the matrix arithmetic operation, the dynamic precision manager to adjust the precision of the compute operation to prevent an arithmetic overflow.

### Claim 2 (depends on 1)

The graphics processing unit as in claim 1, the dynamic precision fixed-point logic of the compute unit including an integer compute unit.

### Claim 3 (depends on 2)

The graphics processing unit as in claim 2, wherein the integer compute unit includes a multiplier, an adder, and accumulator, a shifter, and a register.

### Claim 4 (depends on 3)

The graphics processing unit as in claim 3, wherein the register is to store a dynamic fixed-point scale factor.

### Claim 5 (depends on 4)

The graphics processing unit as in claim 4, the instruction to cause the compute unit to perform a matrix arithmetic operation for a convolution operation on input data to the neural network.

### Claim 6 (depends on 5)

The graphics processing unit as in claim 5, wherein the matrix operation includes an addition or multiplication operation.

### Claim 7 (depends on 6)

The graphics processing unit as in claim 6, wherein the matrix operation includes a multiply and accumulate operation.

### Claim 8 (depends on 7)

The graphics processing unit as in claim 7, the dynamic precision manager to dynamically adjust the precision of the compute operation to prevent an arithmetic overflow at the accumulator.

### Claim 9

A method comprising:
receiving a set of dynamic fixed-point tensors;
computing a right-shift value using an absolute maximum value within the set of dynamic fixed-point tensors and a dynamic range of the set of dynamic fixed-point tensors;
right-shifting data values within the set of dynamic fixed-point tensors;
incrementing a shared exponent associated with the set of dynamic fixed-point tensors based on an amount of right-shift applied to the data values; and
performing a compute operation on the set of dynamic fixed-point tensors.

### Claim 10 (depends on 9)

The method as in claim 9, wherein performing the compute operation on the set of dynamic fixed-point tensors includes:
generating an output tensor via the compute operation on the set of dynamic fixed-point tensors;
determining whether the leading zero count of the absolute max of the output tensor is above a threshold; and
adjusting a precision associated with the output tensor in response to determining whether the leading zero count is above the threshold.

### Claim 11 (depends on 10)

The method as in claim 10, wherein adjusting the precision associated with the output tensor includes adjusting a right-shift counter and shared exponent for the output tensor.

### Claim 12 (depends on 10)

The method as in claim 10, wherein adjusting the precision associated with the output tensor includes increasing the right-shift counter and incrementing a shared exponent in response to determining that the leading count zero is above the threshold.

### Claim 13 (depends on 10)

The method as in claim 10, wherein adjusting the precision associated with the output tensor includes decreasing the right-shift counter and decrementing a shared exponent in response to determining that the leading count zero is below the threshold.

### Claim 14 (depends on 10)

The method as in claim 10, additionally comprising:
performing additional compute operations using the output tensor; and
adjusting the precision associated with the output tensor based on the additional compute operations.

### Claim 15

A data processing system comprising:
a non-transitory machine readable medium storing instructions; and
one or more processors including at least one graphics processor, the at least one graphics processor including:
a compute unit including a hardware logic unit having dynamic precision fixed-point logic, the compute unit to perform a matrix arithmetic operation on a set of dynamic fixed-point tensors; and
a dynamic precision manager to dynamically adjust the precision of a compute operation performed by the compute unit during the matrix arithmetic operation on the set of dynamic fixed-point tensors, the dynamic precision manager to prevent an arithmetic overflow during the compute operation.

### Claim 16 (depends on 15)

The data processing system as in claim 15, the dynamic precision fixed-point logic of the compute unit including an integer compute unit.

### Claim 17 (depends on 16)

The data processing system as in claim 16, wherein the integer compute unit includes a multiplier, an adder, and accumulator, a shifter, and a register.

### Claim 18 (depends on 17)

The data processing system as in claim 17, wherein the register is to store a dynamic fixed-point scale factor.

### Claim 19 (depends on 18)

The data processing system as in claim 18, the instructions of the non-transitory machine readable medium to cause the compute unit to perform a matrix arithmetic operation associated with a convolution operation on input data to a neural network.

### Claim 20 (depends on 15)

The data processing system as in claim 15, wherein to perform a matrix arithmetic operation on the set of dynamic fixed-point tensors includes to:
receive an input tensor associated with the matrix operation;
divide the input tensor into multiple blocks, the multiple blocks having different fixed-point precisions;
determine a shared exponent for each of the multiple blocks;
convert each of the multiple blocks into a dynamic fixed-point format using the shared exponent for each block;
store metadata for the multiple blocks to indicate a data format and shared exponent for the multiple blocks; and
perform the matrix arithmetic operation on the divided dynamic fixed-point input tensors.
