Systems and methods for compressing models during model training while considering target alignment for efficient ai runtime

A computing device, comprising:

• a memory comprising machine executable code;
• and
• a processor coupled to the memory, the processor configured to execute the machine executable code to cause the machine to:
• initiate training of a model for execution on a target device;
• compress, during the training, weights of the model utilizing palettes to represent weight values using bits, wherein the compressing includes adjusting a weight value of a weight that occurs less frequently than other weight values of the weight;
• in response to identifying an infrequently utilized palette that is less frequently repeated than other palettes, implement a coding procedure to either remove the infrequently utilized palette or modify a bit representation for the infrequently utilized palette to be different than bit representations used for the other palettes;
• and
• deploy the model to the target device for execution.

A non-transitory machine readable medium comprising instructions for performing a method, which when executed by a machine, causes the machine to:

• initiate training of a model for execution on a target device;
• compress, during the training, weights of the model utilizing palettes to represent weight values using bits, wherein the compressing includes adjusting a weight value of a weight that occurs less frequently than other weight values of the weight;
• in response to identifying an infrequently utilized palette that is less frequently repeated than other palettes, implement a coding procedure to either remove the infrequently utilized palette or modify a bit representation for the infrequently utilized palette to be different than bit representations used for the other palettes;
• and
• deploy the model to the target device for execution.