Weight quantization adaptation technology

At least one computer readable storage medium comprising a plurality of executable program instructions, which when executed by a computing system, cause the computing system to:

• select a subset of linear layers from a plurality of linear layers in a pre-trained artificial intelligence (AI) model, wherein a quantization error of the subset of linear layers exceeds an error threshold;
• and
• for each linear layer in the subset of linear layers:
• solve a singular value decomposition (SVD) approximation,
• generate a first adapter layer and a second adapter layer based on the SVD approximation, wherein the first adapter layer and the second adapter layer include weight matrices having a first dimension that is less than a first rank threshold and a second dimension that is greater than a second rank threshold, and
• determine an inference output based on the linear layer, the first adapter layer and the second adapter layer.

A semiconductor apparatus comprising:

• one or more substrates;
• and
• logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable or fixed-functionality hardware, the logic to:
• select a subset of linear layers from a plurality of linear layers in a pre-trained artificial intelligence (AI) model, wherein a quantization error of the subset of linear layers exceeds an error threshold;
• and
• for each linear layer in the subset of linear layers:
• solve a singular value decomposition (SVD) approximation,
• generate a first adapter layer and a second adapter layer based on the SVD approximation, wherein the first adapter layer and the second adapter layer include weight matrices having a first dimension that is less than a first rank threshold and a second dimension that is greater than a second rank threshold, and
• determine an inference output based on the linear layer, the first adapter layer and the second adapter layer.