Scaling half-precision floating point tensors for training deep neural networks
AssigneeIntel Corporation
InventorsNAVEEN MELLEMPUDI, DIPANKAR DAS
Life of this patent
Estimating the term…
Abstract
One embodiment provides for a machine-learning accelerator device a multiprocessor to execute parallel threads of an instruction stream, the multiprocessor including a compute unit, the compute unit including a set of functional units, each functional unit to execute at least one of the parallel threads of the instruction stream. The compute unit includes compute logic configured to execute a single instruction to scale an input tensor associated with a layer of a neural network according to a scale factor, the input tensor stored in a floating-point data type, the compute logic to scale the input tensor to enable a data distribution of data of the input tensor to be represented by a 16-bit floating point data type.
Claims20
A graphics processor comprising:
- an interconnect to a fabric interface;
- and
- a graphics processing cluster including a plurality of multiprocessors, the plurality of multiprocessors interconnected via a data crossbar, at least one multiprocessor including a compute circuitry configured to:
- execute at least one single instruction to scale an input tensor associated with a layer of a neural network according to a scale factor, the input tensor stored in a floating-point data type, the input tensor scaled to enable a data distribution of data of the input tensor to be represented by a 16-bit floating point data type.
A method implemented via a machine-learning accelerator, the method comprising:
- executing at least one single instruction to perform a scaled tensor compute operation;
- in response to the at least one single instruction, scaling data of an input tensor associated with a layer of a neural network according to a scale factor, executing a compute operation on scaled data of the input tensor, and re-scaling computed data of the compute operation to generate re-scaled computed data;
- down-converting the re-scaled computed data;
- and
- storing down-converted re-scaled computed data to a 16-bit floating-point data type.
A graphics processing system comprising:
- a memory device;
- and
- a graphics processor comprising a graphics processing cluster including a plurality of multiprocessors, the plurality of multiprocessors interconnected via a data crossbar, at least one multiprocessor including a compute circuitry configured to:
- execute at least one single instruction to scale an input tensor associated with a layer of a neural network according to a scale factor, the input tensor stored in a floating-point data type, the input tensor scaled to enable a data distribution of data of the input tensor to be represented by a 16-bit floating point data type.
Dependencies are inferred from the claim text.