US 2025/0094864 A1 · COMPRESSION OF MACHINE LEARNING MODELS VIA SPARSIFICATION AND QUANTIZATION

Abstract

Machine learning is a process that learns a model from a given dataset, where the model can then be used to make a prediction about new data. In order to reduce the size, computation, and latency of a machine learning model, a compression technique can be employed which includes model sparsification and quantization. To limit the extent to which the quality of the model is impacted when uniformly applying sparsification and quantization to all values of the model, the present disclosure provides for a hybrid sparsification and quantization of the model.

Claims51

Dependent claims

A method, comprising:

at a device, compressing a machine learning model having a plurality of values to reduce at least one of a size of the machine learning model or computation requirements of the machine learning model, by:
processing the machine learning model to generate a plurality of sparse data structures including:
storing inlier values of the machine learning model in a first data structure, and
storing outlier values of the machine learning model in a second data structure,
wherein at least one of the first data structure or the second data structure has a structured sparse pattern;
and
non-uniformly quantizing the machine learning model, including:
quantizing the first data structure storing the inlier values to a first bit width, and
quantizing the second data structure storing the outlier values to a second bit width that is different from the first bit width.

6 more elements

Dependencies are inferred from the claim text.

Compression of machine learning models via sparsification and quantization

Life of this patent

Compression of machine learning models via sparsification and quantization

Life of this patent

Abstract

Claims51