Exponential and logarithmic function approximation for floating point numbers

Abstract

Approaches presented herein provide for the efficient approximation of exponential or logarithmic functions using values in half-precision, floating point format. In at least one embodiment, an activation function of a neural network can perform tasks such as to convert raw scores from the network into probabilities using the exponential of each output. A fixed point representation of each raw value can be generated that includes a bias factor, and used to generate an intermediate representation having a determined number of bits. The intermediate representation can be incremented so that the mantissa corresponds to the fixed point representation. The decimal point can be shifted to obtain a value that is an exponential approximation of the raw value, and used to determine probability (or another such value) for a respective computing operation. At least some of these operations can be reversed to similarly obtain an approximation for a logarithmic function.

Dependencies are inferred from the claim text.