Efficient look-up table based functions for artificial intelligence (AI) accelerator

A system for approximating an activation function, the system comprising:

• a bit shifter configured to receive an input value of the activation function and perform a shift operation on the input value to compute most significant bits (MSBs) of the input value, wherein the system is configured to determine a range of input values that falls in a quadratic interpolation range of the activation function;
• a first device configured to determine if the input value is within the quadratic interpolation range comprising a set of non-uniform intervals, and determine a selected interval from among the set of non-uniform intervals comprising the input value based on the MSBs of the input value;
• based on determining that the input value is within the quadratic interpolation range, a second device configured to retrieve from a look-up table (LUT) associated with the activation function, values of one or more quadratic interpolation parameters associated with the selected interval;
• and
• one or more multiply-accumulate-scale circuits configured to:
• perform a quadratic interpolation on the input value to approximate the input value using the values of the one or more quadratic interpolation parameters, and to determine a first approximated output of the activation function based on a result of the quadratic interpolation performed on the input value;
• and
• based on the first device determining that a second input value is outside the quadratic interpolation range, the one or more multiply-accumulate-scale circuits are further configured to perform a linear extrapolation in one or more linear extrapolation ranges comprising a range from an upper limit value of the quadratic interpolation range to positive infinity and in a range from a lower limit value of the quadratic interpolation ranges to negative infinity by:
• performing the linear extrapolation on the second input value to approximate the activation function at the second input value using an offset value and a slope corresponding to a linear extrapolation range containing the second input value from among a plurality of offset values and slopes of the activation function for each of the linear extrapolation ranges, and
• determining a second approximated output of the activation function based on a result of the linear extrapolation performed on the second input value.

A non-transitory computer readable medium comprising stored representation of an accelerator circuit, which when synthesized by a processor, cause the processor to synthesize a structure of an electronic circuit configured to:

• receive an input value of an activation function;
• determining, by a hardware accelerator, a range of input values that falls in a quadratic interpolation range of the activation function;
• determine, by the hardware accelerator, whether the input value is within the quadratic interpolation range, wherein the quadratic interpolation range comprises a set of non-uniform intervals;
• determine, by the hardware accelerator, a selected interval from among the set of non-uniform intervals comprising the input value;
• based on determining that the input value is within the quadratic interpolation range:
• retrieve, by the hardware accelerator, from a look-up table (LUT) associated with the activation function, values of one or more quadratic interpolation parameters associated with the selected interval;
• perform a quadratic interpolation on the input value to approximate the input value using the values of the one or more quadratic interpolation parameters;
• and
• determine a first approximated output of the activation function based on a result of the quadratic interpolation performed on the input value, and
• based on determining that a second input value is outside the quadratic interpolation range, the hardware accelerator determines to perform a linear extrapolation in one or more linear extrapolation ranges comprising a range from an upper limit value of the quadratic interpolation range to positive infinity and a range from a lower limit value of the quadratic interpolation ranges to negative infinity by:
• determining, offset values and slopes of the activation function for each of the linear extrapolation ranges;
• performing the linear extrapolation on the second input value to approximate the activation function at the second input value using an offset value and a slope corresponding to the linear extrapolation range;
• and
• determining a second approximated output of the activation function based on a result of the linear extrapolation performed on the second input value.