# US 2025/0165854 A1 · Quantization compensation for machine learning models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165854 A1
- Document ID: US20250165854A1
- Status: Published application
- Kind: A1
- Application number: 18/514,602
- Filed: 2023-11-20
- Published: 2025-05-22
- Earliest claimed: 2023-11-20
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Simyung CHANG; Jaeseong YOU; Minseop PARK; Kyuhong SHIM; Kyu Woong HWANG
- Web page: https://patentgrep.com/patent/US20250165854A1

## CPC

- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 20/00: Machine learning
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0499: Feedforward networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N 20/20: Ensemble learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Certain aspects of the present disclosure provide techniques and apparatus for improved machine learning. A first machine learning model comprising a first plurality of blocks is accessed, the first plurality of blocks being associated with a first precision. A second machine learning model comprising a second plurality of blocks associated with a second precision, where the second plurality of blocks comprises a first block that corresponds to a first block of the first plurality of blocks. An input to the first machine learning model is processed using the first plurality of blocks and the second plurality of blocks, comprising modifying an output of the first block of the first plurality of blocks based on the corresponding first block of the second plurality of blocks. An output of the first machine learning model is provided based on the processing.

## Claims

### Claim 1

A processing system comprising:
one or more memories comprising computer-executable instructions; and
one or more processors configured to execute the computer-executable instructions and cause the processing system to:
access a first machine learning model comprising a first plurality of blocks, the first plurality of blocks being associated with a first precision and comprising a first block;
access a second machine learning model comprising a second plurality of blocks associated with a second precision different from the first precision, wherein:
the second plurality of blocks comprises a first block; and
the first block of the second plurality of blocks corresponds to the first block of the first plurality of blocks;
process an input to the quantized machine learning model using the first plurality of blocks of the first machine learning model and the second plurality of blocks of the second machine learning model, wherein, to process the input, the one or more processors are configured to execute the computer-executable instructions and cause the processing system to modify an output of the first block of the first plurality of blocks based on the corresponding first block of the second plurality of blocks; and
provide an output of the first machine learning model based on the processing.

### Claim 2 (depends on 1)

The processing system of claim 1, wherein the second precision is higher than the first precision.

### Claim 3 (depends on 2)

The processing system of claim 2, wherein the second precision corresponds to a 16-bit bit width and wherein the first precision corresponds to a 4-bit bit width.

### Claim 4 (depends on 1)

The processing system of claim 1, wherein:
the first machine learning model has a first size;
the second machine learning model has a second size; and
the second size is smaller than the first size.

### Claim 5 (depends on 1)

The processing system of claim 1, wherein:
the first machine learning model is accessed by a first circuit of an integrated-circuit (IC) device; and
the second machine learning model is accessed by a second circuit of the IC device different from the first circuit.

### Claim 6 (depends on 1)

The processing system of claim 1, wherein:
the first machine learning model was generated by quantizing a baseline machine learning model having a baseline precision higher than the first precision, and
the second machine learning model was trained to adjust for quantization errors resulting from the quantization of the baseline machine learning model.

### Claim 7 (depends on 1)

The processing system of claim 1, wherein:
the first plurality of blocks comprises an ordered network of blocks,
the first plurality of blocks further comprises a second block configured to receive as an input the modified output of the first block of the first plurality of blocks and to process the received input, and
to process the input, the one or more processors are configured to further execute the computer-executable instructions and cause the processing system to modify an output of the second block of the first plurality of blocks using a corresponding second block of the second plurality of blocks.

### Claim 8 (depends on 1)

The processing system of claim 1, wherein:
the first machine learning model was trained based on training data from a source domain,
the second machine learning model was trained using adjustment data from the source domain, and
the second machine learning model was trained without using labels for the adjustment data.

### Claim 9 (depends on 8)

The processing system of claim 8, wherein, to train the second machine learning model, the one or more processors are configured to execute the computer-executable instructions and cause the processing system to generate an adjustment loss for a first block of the second machine learning model based on:
(i) a first feature map generated by a first block of a baseline machine learning model based on a first exemplar in the adjustment data;
(ii) a second feature map generated by a quantized version of the first block of the baseline machine learning model based on the first exemplar, the quantized version corresponding to the first block of the first plurality of blocks; and
(iii) a third feature map generated by the first block of the second plurality of blocks based on the first exemplar.

### Claim 10 (depends on 8)

The processing system of claim 8, wherein to train the second machine learning model, the one or more processors are configured to execute the computer-executable instructions and cause the processing system to generate an adjustment loss for a first block of the second plurality of blocks based on:
(i) a first model output generated by a baseline machine learning model based on a first exemplar in the adjustment data;
(ii) a second model output generated by the first machine learning model based on the first exemplar; and
(iii) a third model output generated by the second machine learning model based on the first exemplar.

### Claim 11 (depends on 8)

The processing system of claim 8, wherein:
the one or more processors are configured to further execute the computer-executable instructions and cause the processing system to adapt the second machine learning model to a target domain based on labeled adaptation data for the target domain, and
the first machine learning model is frozen during adaptation to the target domain.

### Claim 12

A processor-implemented method, comprising:
accessing a first machine learning model comprising a first plurality of blocks, the first plurality of blocks being associated with a first precision and comprising a first block;
accessing a second machine learning model comprising a second plurality of blocks associated with a second precision different from the first precision, wherein:
the second plurality of blocks comprises a first block; and
the first block of the second plurality of blocks corresponds to the first block of the first plurality of blocks;
processing an input to the first machine learning model using the first plurality of blocks of the first machine learning model and the second plurality of blocks of the second machine learning model, wherein the processing comprises modifying an output of the first block of the first plurality of blocks based on the corresponding first block of the second plurality of blocks; and
providing an output of the first machine learning model based on the processing.

### Claim 13 (depends on 12)

The processor-implemented method of claim 12, wherein the second precision is higher than the first precision.

### Claim 14 (depends on 13)

The processor-implemented method of claim 13, wherein the second precision corresponds to a 16-bit bit width and wherein the first precision corresponds to a 4-bit bit width.

### Claim 15 (depends on 12)

The processor-implemented method of claim 12, wherein:
the first machine learning model has a first size;
the second machine learning model has a second size; and
the second size is smaller than the first size.

### Claim 16 (depends on 12)

The processor-implemented method of claim 12, wherein:
the first machine learning model is accessed by a first circuit of an integrated-circuit (IC) device; and
the second machine learning model is accessed by a second circuit of the IC device different from the first circuit.

### Claim 17 (depends on 12)

The processor-implemented method of claim 12, wherein:
the first machine learning model was generated by quantizing a baseline machine learning model having a baseline precision higher than the first precision, and
the second machine learning model was trained to adjust for quantization errors resulting from the quantization of the baseline machine learning model.

### Claim 18 (depends on 12)

The processor-implemented method of claim 12, wherein:
the first plurality of blocks comprises an ordered network of blocks,
the first plurality of blocks further comprises a second block configured to receive as an input the modified output of the first block of the first plurality of blocks and to process the received input, and
the processing of the input using the first machine learning model further comprises modifying an output of the second block of the first plurality of blocks using a corresponding second block of the second plurality of blocks.

### Claim 19 (depends on 12)

The processor-implemented method of claim 12, wherein:
the first machine learning model was trained based on training data from a source domain,
the second machine learning model was trained using adjustment data from the source domain, and
the second machine learning model was trained without using labels for the adjustment data.

### Claim 20 (depends on 19)

The processor-implemented method of claim 19, wherein training the second machine learning model comprises generating an adjustment loss for a first block of the second machine learning model based on:
(i) a first feature map generated by a first block of a baseline machine learning model based on a first exemplar in the adjustment data;
(ii) a second feature map generated by a quantized version of the first block of the baseline machine learning model based on the first exemplar, the quantized version corresponding to the first block of the first plurality of blocks; and
(iii) a third feature map generated by the first block of the second plurality of blocks based on the first exemplar.

### Claim 21 (depends on 19)

The processor-implemented method of claim 19, wherein training the second machine learning model comprises generating an adjustment loss for a first block of the second plurality of blocks based on:
(i) a first model output generated by a baseline machine learning model based on a first exemplar in the adjustment data;
(ii) a second model output generated by the first machine learning model based on the first exemplar; and
(iii) a third model output generated by the second machine learning model based on the first exemplar.

### Claim 22 (depends on 19)

The processor-implemented method of claim 19, further comprising adapting the second machine learning model to a target domain based on labeled adaptation data for the target domain, wherein the first machine learning model is frozen during adaptation to the target domain.

### Claim 23

A processor-implemented method, comprising:
accessing a first machine learning model comprising a first plurality of blocks;
generating a second machine learning model comprising a second plurality of blocks by quantizing the first machine learning model;
training a third machine learning model comprising a third plurality of blocks for adjusting for the quantization of the first machine learning model; and
deploying the second machine learning model and the third machine learning model for inferencing.

### Claim 24 (depends on 23)

The processor-implemented method of claim 23, wherein each of the first plurality of blocks comprises at least one of a layer of the first machine learning model or a transformer of the first machine learning model.

### Claim 25 (depends on 23)

The processor-implemented method of claim 23, wherein:
the first machine learning model was trained based on training data from a source domain,
the third machine learning model is trained using adjustment data from the source domain, and
the third machine learning model is trained without using labels for the adjustment data.

### Claim 26 (depends on 25)

The processor-implemented method of claim 25, wherein training the third machine learning model comprises generating an adjustment loss for a first block of the third plurality of blocks based on:
(i) a first feature map generated by a first block of the first plurality of blocks based on a first exemplar in the adjustment data;
(ii) a second feature map generated by a first block of the second plurality of blocks based on the first exemplar, wherein the first block from the second plurality of blocks comprises a quantized version of the first block from the first plurality of blocks; and
(iii) a third feature map generated by the first block of the third plurality of blocks based on the first exemplar, wherein the first block of the third plurality of blocks corresponds to the first block of the second plurality of blocks.

### Claim 27 (depends on 25)

The processor-implemented method of claim 25, wherein training the third machine learning model comprises generating an adjustment loss for a first block of the third plurality of blocks based on:
(i) a first model output generated by the first machine learning model based on a first exemplar in the adjustment data;
(ii) a second model output generated by the second machine learning model based on the first exemplar; and
(iii) a third model output generated by the third machine learning model based on the first exemplar.

### Claim 28 (depends on 25)

The processor-implemented method of claim 25, further comprising adapting the third machine learning model to a target domain based on labeled adaptation data for the target domain, wherein the second machine learning model is frozen during adaptation to the target domain.

### Claim 29 (depends on 23)

The processor-implemented method of claim 23, wherein:
parameters of the second machine learning model are encoded using a first value representation,
parameters of the third machine learning model are encoded using a second value representation, and
the second value representation has a higher precision than the first value representation.

### Claim 30 (depends on 23)

The processor-implemented method of claim 23, wherein deploying the second machine learning model and the third machine learning model for inferencing comprises deploying the second machine learning model to be executed on a first hardware component and deploying the third machine learning model to be executed on a second hardware component.
