Neural network processing using mixed-precision data representation

A computer program product embodied in a non-transitory computer readable medium for data manipulation, the computer program product comprising code which causes one or more processors to perform operations of:

• accessing a first input data element and a second input data element for manipulation on a processor that supports single precision operations, wherein:
• the first input data element comprises one or more parameters of a neural network trained on training data represented by a single-precision data representation, wherein each parameter of the neural network comprises a 16-bit reduced floating point representation that does not vary from parameter to parameter of the neural network, and wherein training the neural network using the single-precision training data produces the parameters of the neural network as represented by a 16-bit reduced floating point representation;
• and
• the second input data element comprises input data, that does not define any parameter of the neural network, for processing by the neural network;
• converting the second input data element from a first data representation to a 16-bit reduced floating point representation;
• processing, by at least a first layer of the neural network, the first and second input data elements, comprising:
• manipulating, by the processor, the first input data element with the second input data element, wherein the first input data element comprises a 16-bit reduced floating point representation, to produce a single-precision data output as a result of the manipulation of the 16-bit reduced floating point first input data element and the 16-bit reduced floating point second input data element;
• outputting the result of the manipulation, wherein the result comprises a single-precision data representation element;
• and
• forwarding the result to a next layer of the neural network, based on the outputting;
• determining whether the next layer of the neural network is an output layer of the neural network;
• in response to a determination that the next layer is not the output layer of the neural network, then defining the second input data element as the result of the manipulation and repeating the accessing, converting, and processing steps with the next layer of the neural network;
• and
• in response to a determination that the next layer is the output layer, then outputting the result of the manipulation.

A computer system for data manipulation comprising:

• a memory which stores instructions;
• and one or more processors attached to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to:
• access a first input data element and a second input data element for manipulation on a processor that supports single precision operations, wherein:
• the first input data element comprises one or more parameters of a neural network trained on training data represented by a single-precision data representation, wherein each parameter of the neural network comprises a 16-bit reduced floating point representation that does not vary from parameter to parameter of the neural network, and wherein training the neural network using the single-precision training data produces the parameters of the neural network as represented by a 16-bit reduced floating point representation;
• and
• the second input data element comprises input data that does not define any parameter of the neural network, for processing by the neural network;
• convert the second input data element from a first data representation to a 16-bit reduced floating point representation;
• process, by at least a first layer of the neural network, the first and second input data elements, comprising:
• manipulating, by the processor, the first input data element with the second input data element, wherein the first input data element comprises a 16-bit reduced floating point representation, to produce a single-precision data output as a result of the manipulation of the 16-bit reduced floating point first input data element and the 16-bit reduced floating point second input data element;
• outputting the result of the manipulation, wherein the result comprises a single-precision data representation element;
• and
• forwarding the result to a next layer of the neural network, based on the outputting;
• determine whether the next layer of the neural network is an output layer of the neural network;
• in response to a determination that the next layer is not the output layer of the neural network, then define the second input data element as the result of the manipulation and repeat the accessing, converting, and processing steps with the next layer of the neural network;
• and
• in response to a determination that the next layer is the output layer, then output the result of the manipulation.