Very low precision floating point representation for deep learning acceleration

A computer usable program product comprising a computer-readable storage device, and program instructions stored on the storage device, the stored program instructions comprising:

• program instructions to configure a specialized circuit for floating point computations using numbers represented by a very low precision format (VLP format), wherein the VLP format comprises less than sixteen bits, and wherein the specialized circuit performs training computations for a neural network;
• program instructions to further configure the specialized circuit to apportion the VLP format into a sign bit, exponent bits (e), and mantissa bits (p);
• program instructions to operate the further configured specialized circuit to store an approximation of a numeric value in the VLP format, wherein the approximation is represented as a function of a multiple of a fraction, wherein the fraction is an inverse of a number of discrete values that can be represented using only the mantissa bits;
• and
• additionally configuring the specialized circuit to represents, in the VLP format, infinity in a same manner as a “Not a Number” (NaN) value.

A computer system comprising a processor, a computer-readable memory, and a computer-readable storage device, and program instructions stored on the storage device for execution by the processor via the memory, the stored program instructions comprising:

• program instructions to configure a specialized circuit for floating point computations using numbers represented by a very low precision format (VLP format), wherein the VLP format comprises less than sixteen bits, and wherein the specialized circuit performs training computations for a neural network;
• program instructions to further configure the specialized circuit to apportion the VLP format into a sign bit, exponent bits (e), and mantissa bits (p);
• program instructions to operate the further configured specialized circuit to store an approximation of a numeric value in the VLP format, wherein the approximation is represented as a function of a multiple of a fraction, wherein the fraction is an inverse of a number of discrete values that can be represented using only the mantissa bits;
• and
• additionally configuring the specialized circuit to represents, in the VLP format, infinity in a same manner as a “Not a Number” (NaN) value.