Method and machine learning system to perform quantization of neural network

A system to perform quantization of a neural network having multiple layers, the apparatus comprises:

• a processor configured to:
• a) receive a floating-point dataset as input dataset at a first layer of the neural network;
• b) determine a first shift constant for the first layer of the neural network based on the input dataset;
• c) perform quantization for the first layer using the determined shift constant of the first layer;
• d) determine a next shift constant for next layer of the neural network based on output of a layer previous to the next layer;
• e) perform quantization for the next layer using the determined next shift constant;
• and
• f) iterate the steps b) and c) for all layers of the neural network, to generate a fixed point dataset as output dataset;
• wherein the floating point dataset is a reduced dataset, wherein the processor is configured to obtain the reduced dataset by steps comprising:
• receiving an original validation dataset as input dataset, wherein the original validation dataset includes a plurality of test images;
• computing an effective vector for each of the plurality of test images, wherein the effective vector is computed from at least one of ratio of maximum value of all test images for each layer of the neural network and maximum value of a particular test image for each layer, and ratio of minimum value of all test images for each layer of the neural network and minimum value of a particular test image for each layer;
• applying k-means clustering and stratified sampling to the computed effective vector of each test images;
• and
• selecting one or more images from each cluster, to obtain the reduced dataset.

A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to:

• a) receive a floating-point dataset as input dataset at a first layer of the neural network;
• b) determine a first shift constant for the first layer of the neural network based on the input dataset;
• c) perform quantization for the first layer using the determined shift constant of the first layer;
• d) determine a next shift constant for next layer of the neural network based on output of a layer previous to the next layer;
• e) perform quantization for the next layer using the determined next shift constant;
• and
• f) iterate the steps b) and c) for all layers of the neural network, to generate a fixed point dataset as output dataset;
• wherein the floating point dataset is a reduced dataset, wherein the processor is configured to obtain the reduced dataset by steps comprising:
• receiving an original validation dataset as input dataset, wherein the original validation dataset includes a plurality of test images;
• computing an effective vector for each of the plurality of test images, wherein the effective vector is computed from at least one of ratio of maximum value of all test images for each layer of the neural network and maximum value of a particular test image for each layer, and ratio of minimum value of all test images for each layer of the neural network and minimum value of a particular test image for each layer;
• applying k-means clustering and stratified sampling to the computed effective vector of each test images;
• and
• selecting one or more images from each cluster, to obtain the reduced dataset.