System and method enabling one-hot neural networks on a machine learning compute platform
AssigneeIntel Corporation
InventorsJianguo Li, Yurong Chen
Life of this patent
Estimating the term…
Abstract
One embodiment provides for a graphics processor comprising a cache memory and a graphics core coupled with the cache memory. The graphics core includes circuitry configured to generate an approximate weight matrix including a set of one-hot coded weights, perform a forward compute pass with mini batch samples to compute a loss function, perform a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update, and update the approximate weight matrix based on the gradient update to generate an updated weight matrix.
Claims20
A graphics processing unit comprising:
- a cache memory;
- and
- a graphics core coupled with the cache memory, the graphics core including circuitry having an instruction set architecture with support for operands in a one-hot encoding format, the circuitry configured to:
- generate an approximate weight matrix including a set of one-hot coded weights;
- perform a forward compute pass with mini batch samples to compute a loss function via execution of an instruction to perform a matrix operation, the instruction having an operand including at least a portion of the set of one-hot coded weights;
- perform a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update;
- and
- update the approximate weight matrix based on the gradient update to generate an updated weight matrix.
A method of performing machine learning operations, the method comprising:
- generating an approximate weight matrix including a set of one-hot coded weights via a graphics processing unit including circuitry having an instruction set architecture with support for operands in a one-hot encoding format;
- performing a forward compute pass with mini batch samples to compute a loss function, including executing an instruction via the graphics processing unit to perform a matrix operation, the instruction having an operand including at least a portion of the set of one-hot coded weights;
- performing a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update;
- and
- updating the approximate weight matrix based on the gradient update to generate an updated weight matrix.
A data processing system comprising:
- a memory device;
- and
- a graphics processing unit coupled with the memory device, the graphics processing unit comprising a graphics core including circuitry having an instruction set architecture with support for operands in a one-hot encoding format, the circuitry configured to:
- generate an approximate weight matrix including a set of one-hot coded weights;
- perform a forward compute pass with mini batch samples to compute a loss function via execution of an instruction to perform a matrix operation, the instruction having an operand including at least a portion of the set of one-hot coded weights;
- perform a backward compute pass to compute a gradient update via stochastic gradient descent according to a loss update;
- and
- update the approximate weight matrix based on the gradient update to generate an updated weight matrix.
Dependencies are inferred from the claim text.