# US 2025/0028978 A1 · Autocontrastive Decoding Among Model Layers

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0028978 A1
- Document ID: US20250028978A1
- Status: Published application
- Kind: A1
- Application number: 18/224,497
- Filed: 2023-07-20
- Published: 2025-01-23
- Earliest claimed: 2023-07-20
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Ariel Gera; RONI FRIEDMAN-MELAMED; Ofir Arviv; Benjamin Sznajder; Chulaka Gunasekara; Eyal Shnarch; Noam Slonim
- Web page: https://patentgrep.com/patent/US20250028978A1

## CPC

- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06N 3/045: Combinations of networks
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Techniques for autocontrastive decoding of a machine learning model are provided. In one aspect, a system for machine learning includes: a multi-layer machine learning model; and an autocontrastive decoding module configured to obtain prediction probabilities from multiple, different layers of the multi-layer machine learning model as data propagates through the multi-layer machine learning model, and aggregate in a contrastive manner the prediction probabilities from the multiple, different layers of the multi-layer machine learning model to provide a final output from the multi-layer machine learning model. The multi-layer machine learning model can be a transformer-based machine learning model. The autocontrastive decoding module can be configured to redistribute a prediction probability distribution of the transformer-based machine learning model by maximizing a difference between log-probabilities of a final layer and one or more intermediate layers of the transformer-based machine learning model. A machine learning method using the present system is also provided.

## Claims

### Claim 1

A system for machine learning, the system comprising:
a multi-layer machine learning model; and
an autocontrastive decoding module configured to obtain prediction probabilities from multiple, different layers of the multi-layer machine learning model as data propagates through the multi-layer machine learning model, and aggregate in a contrastive manner the prediction probabilities from the multiple, different layers of the multi-layer machine learning model to provide a final output from the multi-layer machine learning model.

### Claim 2 (depends on 1)

The system of claim 1, wherein the multi-layer machine learning model comprises one or more intermediate layers and a final output layer.

### Claim 3 (depends on 2)

The system of claim 2, wherein the autocontrastive decoding module is further configured to obtain first prediction probabilities from at least one of the intermediate layers, and obtain second prediction probabilities from the final output layer.

### Claim 4 (depends on 3)

The system of claim 3, wherein the autocontrastive decoding module is further configured to aggregate the first prediction probabilities from at least one of the intermediate layers with the second prediction probabilities from the final output layer, while changing the second prediction probabilities from the final output layer according to the first prediction probabilities obtained from at least one of the intermediate layers.

### Claim 5 (depends on 1)

The system of claim 1, wherein the multi-layer machine learning model comprises a transformer-based machine learning model.

### Claim 6 (depends on 5)

The system of claim 5, wherein at least one of the intermediate layers comprises a linear amateur head, and wherein the final output layer comprises a linear expert head, making the transformer-based machine learning model a multi-exit model.

### Claim 7 (depends on 6)

The system of claim 6, wherein the linear amateur head is configured to map an output of the at least one of the intermediate layers to a first next-token probability distribution, and wherein the linear expert head is configured to map an output of the final output layer to a second next-token probability distribution.

### Claim 8 (depends on 7)

The system of claim 7, wherein the autocontrastive decoding module is further configured to contrast the first next-token probability distribution with the second next-token probability distribution to provide a token-level probability distribution as the final output from the transformer-based machine learning model.

### Claim 9

A system for machine learning, the system comprising:
a transformer-based machine learning model; and
an autocontrastive decoding module configured to redistribute a prediction probability distribution of the transformer-based machine learning model by maximizing a difference between log-probabilities of a final layer and one or more intermediate layers of the transformer-based machine learning model.

### Claim 10 (depends on 9)

The system of claim 9, wherein at least one of the intermediate layers comprises a linear amateur head, and wherein the final output layer comprises a linear expert head, making the transformer-based machine learning model a multi-exit model.

### Claim 11 (depends on 10)

The system of claim 10, wherein the linear amateur head is configured to map an output of the at least one of the intermediate layers to a first next-token probability distribution, and wherein the linear expert head is configured to map an output of the final output layer to a second next-token probability distribution.

### Claim 12 (depends on 11)

The system of claim 11, wherein the autocontrastive decoding module is further configured to contrast the first next-token probability distribution with the second next-token probability distribution to provide a token-level probability distribution for the transformer-based machine learning model.

### Claim 13

A machine learning method, comprising:
providing data as input to a multi-layer machine learning model;
obtaining prediction probabilities from multiple, different layers of the multi-layer machine learning model as the data propagates through the multi-layer machine learning model; and
aggregating in a contrastive manner the prediction probabilities from the multiple, different layers of the multi-layer machine learning model to provide a final output from the multi-layer machine learning model.

### Claim 14 (depends on 13)

The machine learning method of claim 13, wherein the multi-layer machine learning model comprises one or more intermediate layers and a final output layer.

### Claim 15 (depends on 14)

The machine learning method of claim 14, further comprising:
obtaining first prediction probabilities from at least one of the intermediate layers; and
obtaining second prediction probabilities from the final output layer.

### Claim 16 (depends on 15)

The machine learning method of claim 15, further comprising:
aggregating the first prediction probabilities from at least one of the intermediate layers with the second prediction probabilities from the final output layer, while changing the second prediction probabilities from the final output layer according to the first prediction probabilities obtained from at least one of the intermediate layers.

### Claim 17 (depends on 14)

The machine learning method of claim 14, wherein the multi-layer machine learning model comprises a transformer-based machine learning model.

### Claim 18 (depends on 17)

The machine learning method of claim 17, further comprising:
adding a linear head to at least one of the intermediate layers to make the transformer-based machine learning model a multi-exit model.

### Claim 19 (depends on 17)

The machine learning method of claim 17, further comprising:
redistributing a prediction probability distribution of the transformer-based machine learning model by maximizing a difference between log-probabilities of the final output layer and at least one of the intermediate layers.

### Claim 20 (depends on 19)

The machine learning method of claim 19, further comprising:
providing the prediction probability distribution of the transformer-based machine learning model which has been redistributed as the final output from the multi-layer machine learning model.
