Approximating activation functions in neural networks with programmable look-up table

A method for executing a neural network, the method comprising:

• receiving an input data element of a function in the neural network;
• storing, in a look-up table, one or more parameters of an approximation of the function over a range of input data elements of the function;
• determining whether the input data element falls into the range of input data elements;
• and
• in response to determining that the input data element falls into the range of input data elements:
• determining an address of at least one parameter of the function based on the input data element,
• retrieving the one or more parameters of the function from the look-up table based on the address, and
• computing an output data element of the function based on the received one or more parameters of the function and the input data element.

One or more non-transitory computer-readable media storing instructions executable to perform operations for executing a non-linear activation function in a neural network, the operations comprising:

• receiving an input data element of a function in the neural network;
• storing, in a look-up table, one or more parameters of an approximation of the function over a range of input data elements of the function;
• determining whether the input data element falls into the range of input data elements;
• and
• in response to determining that the input data element falls into the range of input data elements:
• determining an address of at least one parameter of the function based on the input data element,
• retrieving the one or more parameters of the function from the look-up table based on the address, and
• computing an output data element of the function based on the received one or more parameters of the function and the input data element.