Iterative normalization for machine learning applications

A semiconductor package apparatus, comprising:

• one or more substrates;
• and
• logic coupled to the one or more substrates, wherein the logic is at least partly implemented in one or more of configurable logic or fixed-functionality logic hardware, the logic coupled to the one or more substrates to:
• process one or more vectors with a sum of squares operation with a layer of a multi-layer neural network,
• iteratively sum squares of first elements of the one or more vectors until a square of a current element of the first elements during a current iteration exceeds a threshold value, wherein the summation of squares of the first elements is to generate a first value;
• iteratively sum squares of second elements of the one or more vectors, wherein the second elements are different from the first elements, wherein the summation of squares of the second elements is to generate a second value;
• determine a fixed-point approximation for the sum of squares operation based on the first value and the second value so as to determine the fixed-point approximation with a fixed-point logic circuit that is to bypass one or more overflow operations, wherein the multi-layer neural network is further to provide one or more of supervised learning or unsupervised learning for a machine learning application based at least in part on the fixed-point approximation.

A method of machine learning, comprising:

• processing one or more vectors with a sum of squares operation with a layer of a multi-layer neural network;
• iteratively summing squares of first elements of the one or more vectors until a square of a current element of the first elements during a current iteration exceeds a threshold value, wherein the summing of squares of the first elements generates a first value;
• iteratively summing squares of second elements of the one or more vectors, wherein the second elements are different from the first elements, wherein the summing of squares of the second elements generates a second value;
• and
• determining a fixed-point approximation for the sum of squares operation based on the first value and the second value so as to determine the fixed-point approximation with a fixed-point logic circuit that is to bypass one or more overflow operations, wherein the multi-layer neural network further provides one or more of supervised learning or unsupervised learning for a machine learning application based at least in part on the fixed-point approximation.

At least one non-transitory computer readable storage medium, comprising a set of instructions, which when executed by a computing device, cause the computing device to:

• process one or more vectors with a sum of squares operation with a layer of a multi-layer neural network;
• iteratively sum squares of first elements of the one or more vectors until a square of a current element of the first elements during a current iteration exceeds a threshold value, wherein the summation of squares of the first elements is to generate a first value;
• iteratively sum squares of second elements of the one or more vectors, wherein the second elements are different from the first elements, wherein the summation of squares of the second elements is to generate a second value;
• and
• determine a fixed-point approximation for the sum of squares operation based on the first value and the second value so as to determine the fixed-point approximation with a fixed-point logic circuit that is to bypass one or more overflow operations, wherein the multi-layer neural network is further to provide one or more of supervised learning or unsupervised learning for a machine learning application based at least in part on the fixed-point approximation.