Selective and flexible compression of data associated with a machine learning model

A system comprising:

• one or more processors;
• and
• first memory coupled to the one or more processors, the first memory comprising instructions executable by the one or more processors, the one or more processors being operable when executing the instructions to:
• compress first data based on a first compression scheme to generate second data, wherein the first data is associated with a first machine learning model;
• store the second data into a second memory;
• adjust a first entry of a lookup table to correspond to the first compression scheme based on the first data being compressed based on the first compression scheme;
• provide the second data from the second memory to processing elements of a processing array during execution of the first machine learning model;
• decompress, at the processing array, the second data based on the lookup table to obtain the first data;
• determine that compression of the first data with the first compression scheme reduces a size of the first data to be less than or equal to an available memory space of the second memory;
• predict that the first compression scheme is to be applied at least a first number of times;
• determine that the at least the first number of times meets a reuse threshold;
• select the first compression scheme based on the at least the first number of times meeting the reuse threshold, and compression of the first data with the first compression scheme reducing the size of the first data to be less than or equal to the available memory space;
• generate metadata to associate the first compression scheme with the second data;
• and
• store the second data in association with the metadata into the second memory.

A method comprising:

• compressing first data based on a first compression scheme to generate second data, wherein the first data is associated with a first machine learning model;
• storing the second data into a memory;
• adjusting a first entry of a lookup table to correspond to the first compression scheme based on the first data being compressed based on the first compression scheme;
• providing the second data from the memory to processing elements of a processing array during execution of the first machine learning model;
• and
• decompressing, at the processing array, the second data based on the lookup table to obtain the first data;
• wherein:
• the first data includes one or more of weights or biases;
• the first compression scheme is an Asymmetric Numeral Systems compression scheme;
• the first data is to be compressed based on the first compression scheme prior to the first machine model being executed;
• and to decompress the second data, the processing array is to decompress the second data with the processing elements,
• wherein the method, further comprising:
• determining that compression of the first data with the first compression scheme reduces a size of the first data to be less than or equal to an available memory space of the memory;
• predicting that the first compression scheme is to be applied at least a first number of times;
• determining that the at least the first number of times meets a reuse threshold;
• selecting the first compression scheme based on the at least the first number of times meeting the reuse threshold, and compression of the first data with the first compression scheme reducing the size of the first data to be less than or equal to the available memory space;
• generating metadata to associate the first compression scheme with the second data;
• and
• storing the second data in association with the metadata into the memory.