Neural network output layer for machine learning

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network;
• and
• the layer is based on a parabolic estimator function.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network,
• the layer is based on a parabolic estimator function;
• and
• the parabolic estimator function is implemented using lookup tables.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network,
• the layer is based on a parabolic estimator function;
• the parabolic estimator function is implemented using lookup tables;
• and
• the lookup tables comprise three lookup tables.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network,
• the layer is based on a parabolic estimator function;
• the parabolic estimator function is implemented using lookup tables;
• the lookup tables comprise three lookup tables;
• and
• further comprising two multiplex per element of an output vector.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network,
• the layer is based on a parabolic estimator function;
• the parabolic estimator function is implemented using lookup tables;
• the lookup tables comprise three lookup tables;
• further comprising two multiplex per element of an output vector;
• and
• no more than three multipliers per element of an output vector are used.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network,
• the layer is based on a parabolic estimator function;
• the parabolic estimator function is implemented using lookup tables;
• the lookup tables comprise three lookup tables;
• and
• the three lookup tables are each 256 elements or fewer in length.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network,
• the layer is based on a parabolic estimator function;
• the parabolic estimator function is implemented using lookup tables;
• and
• the lookup tables comprise equidistant points.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network,
• the layer is based on a parabolic estimator function;
• the parabolic estimator function is implemented using lookup tables;
• the lookup tables comprise equidistant points;
• and
• values between two equidistant points with a lookup table correspond to an integer number of intervals between the two points.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network,
• the layer is based on a parabolic estimator function;
• the parabolic estimator function is implemented using lookup tables;
• the lookup tables comprise equidistant points;
• values between two equidistant points with a lookup table correspond to an integer number of intervals between the two points;
• and
• the integer number of intervals is based on a power of two.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network;
• the layer is based on a parabolic estimator function;
• and
• the parabolic estimator function is implemented using one lookup table.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network;
• the layer is based on a parabolic estimator function;
• and
• the parabolic estimator function is based on a moving set of three points.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network;
• the layer is based on a parabolic estimator function;
• and
• the parabolic estimator function implements eXi or kXi for the layer, where Xi are arguments of the Softmax function, k is a real number greater than 1, and e is the natural logarithm base.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network;
• the layer is based on a parabolic estimator function;
• the parabolic estimator function implements eXi or kXi for the layer, where Xi are arguments of the Softmax function, k is a real number greater than 1, and e is the natural logarithm base;
• and
• kXi is calculated using 16-bit fixed-point arithmetic.

A processor-implemented method for data manipulation comprising:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector;
• wherein the layer that maps the first vector comprises a Softmax function,
• the layer that maps the first vector is an output layer for the neural network;
• the layer is based on a parabolic estimator function;
• the parabolic estimator function implements eXi or kXi for the layer, where Xi are arguments of the Softmax function, k is a real number greater than 1, and e is the natural logarithm base;
• and
• k is equal to 2.

A computer program product embodied in a non-transitory computer readable medium for data manipulation, the computer program product comprising code which causes one or more processors to perform operations of:

• configuring a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implementing a layer, based on a parabolic estimator function,
• within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums of one using fixed point calculations;
• and
• classifying results of the neural network based on a value of the second vector.

A computer system for data manipulation comprising:

• a memory which stores instructions;
• one or more processors coupled to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to
• configure a plurality of processing elements within a reconfigurable fabric to implement a data flow graph, wherein the data flow graph implements a neural network;
• implement a layer, based on a parabolic estimator function,
• within the neural network, that maps a first vector of real values to a second vector of real values bounded by zero and one wherein the second vector sums to a value of one using fixed point calculations;
• and
• classify results of the neural network based on a value of the second vector.