Deep neural network architecture using piecewise linear approximation
AssigneeIntel Coroporation
InventorsKamlesh Pillai, Gurpreet S. Kalsi, Amit Mishra
Life of this patent
Estimating the term…
Abstract
A log circuit for piecewise linear approximation is disclosed. The log circuit identifies an input associated with a logarithm operation to be performed using piecewise linear approximation. The log circuit then identifies a range that the input falls within from various ranges associated with piecewise linear approximation (PLA) equations for the logarithm operation, where the identified range corresponds to one of the PLA equations. The log circuit computes a result of the corresponding PLA equation based on the respective operands of the equation. The log circuit then returns an output associated with the logarithm operation, which is based at least partially on the result of the PLA equation.
Claims22
An activation function circuit, comprising:
- decode circuitry to decode a first instruction to perform a first activation function, wherein the first activation function is one of a plurality of activation functions implemented on the activation function circuit, and wherein the first activation function includes an exponent operation and a division operation;
- exponent circuitry to perform the exponent operation using piecewise linear approximation;
- log circuitry to perform logarithm operations on a numerator and a denominator of the division operation using piecewise linear approximation;
- subtractor circuitry to perform a subtraction operation on results of the logarithm operations;
- antilog circuitry to perform an antilogarithm operation on a result of the subtraction operation using piecewise linear approximation;
- and
- output circuitry to output a result of the first activation function based on a result of the antilogarithm operation.
At least one non-transitory machine-accessible storage medium having instructions stored thereon, wherein the instructions, when implemented or executed on processing circuitry comprising exponent circuitry, log circuitry, antilog circuitry, and subtractor circuitry, cause the processing circuitry to:
- receive a first instruction to perform a first activation function, wherein the first activation function is one of a plurality of activation functions implemented on the processing circuitry, and wherein the first activation function includes an exponent operation and a division operation;
- perform, using the exponent circuitry, the exponent operation using piecewise linear approximation;
- perform, using the log circuitry, logarithm operations on a numerator and a denominator of the division operation using piecewise linear approximation;
- perform, using the subtractor circuitry, a subtraction operation on results of the logarithm operations;
- perform, using the antilog circuitry, an antilogarithm operation on a result of the subtraction operation using piecewise linear approximation;
- and
- output a result of the first activation function based on a result of the antilogarithm operation.
A system, comprising:
- a processor;
- and
- an activation function circuit, comprising:
- decode circuitry to decode a first instruction to perform a first activation function, wherein the first activation function is one of a plurality of activation functions implemented on the activation function circuit, and wherein the first activation function includes an exponent operation and a division operation;
- exponent circuitry to perform the exponent operation using piecewise linear approximation;
- log circuitry to perform logarithm operations on a numerator and a denominator of the division operation using piecewise linear approximation;
- subtractor circuitry to perform a subtraction operation on results of the logarithm operations;
- antilog circuitry to perform an antilogarithm operation on a result of the subtraction operation using piecewise linear approximation;
- and
- output circuitry to output a result of the first activation function based on a result of the antilogarithm operation.
Dependencies are inferred from the claim text.