Neural network inference quantization

A computer program product comprising:

• one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the stored program instructions comprising:
• program instructions to responsive to neural network run-time, reduce one or more sets of maximum activations along a hidden dimension respectively associated with one or more activation tensors and one or more layers of a neural network;
• program instructions to compute an interquartile range (IQR) clip threshold for each reduced set for each sequence dimension in the neural network;
• program instructions to clip one or more activations based on respective computed IQR clip thresholds;
• and
• program instructions to quantize the clipped activations.

A computer system comprising:

• one or more computer processors;
• one or more computer readable storage media;
• and
• program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the stored program instructions comprising:
• program instructions to responsive to neural network run-time, reduce one or more sets of maximum activations along a hidden dimension respectively associated with one or more activation tensors and one or more layers of a neural network;
• program instructions to compute an interquartile range (IQR) clip threshold for each reduced set for each sequence dimension in the neural network;
• program instructions to clip one or more activations based on respective computed IQR clip thresholds;
• and
• program instructions to quantize the clipped activations.