Activation accelerator for neural network accelerator

A method of applying a non-linear activation function to an input value using an activation accelerator, the method comprising:

• receiving, at the activation accelerator, an input value, an input offset, and an absolute value flag;
• generating, at the activation accelerator, a converted value from the input value, wherein, when the absolute value flag is not set, generating the converted value comprises combining the input value and the input offset, and, when the absolute value flag is set, generating the converted value comprises generating an absolute value of the combination of the input value and the input offset;
• generating, at the activation accelerator, an index and an interpolation value from the converted value;
• retrieving multiple values from a look-up table of the activation accelerator based on the index, the look-up table configured to store a plurality of values representing the non-linear activation function;
• and
• generating, at the activation accelerator, an estimated result of the non-linear activation function for the input value from an interpolation output generated by interpolating between the multiple values retrieved from the look-up table based on the interpolation value.

A non-transitory computer readable storage medium having stored thereon a computer readable dataset description of an activation accelerator for use in a neural network accelerator that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying an activation accelerator comprising:

• a look-up table configured to store a plurality of values representing a non-linear activation function;
• and
• one or more activation pipelines, each activation pipeline comprising:
• a range conversion unit configured to:
• receive an input value, an input offset, and an absolute value flag, and
• generate a converted value from the input value, by, when the absolute value flag is not set, combining the input value and the input offset, and, when the absolute value flag is set, generating an absolute value of the combination of the input value and the input offset;
• an index generation unit configured to generate an index and an interpolation value from the converted value;
• a look-up table interface unit configured to retrieve multiple values from the look-up table based on the index;
• and
• an interpolation unit configured to generate an estimate of a result of the non-linear activation function for the input value from an interpolation output generated by interpolating between the multiple values retrieved from the look-up table based on the interpolation value.