# US 2025/0165792 A1 · Adversarial training of machine learning models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165792 A1
- Document ID: US20250165792A1
- Status: Published application
- Kind: A1
- Application number: 19/034,250
- Filed: 2025-01-22
- Published: 2025-05-22
- Earliest claimed: 2019-11-07
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Xiaodong LIU; Jianfeng GAO; Pengcheng HE; Weizhu CHEN
- Web page: https://patentgrep.com/patent/US20250165792A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/045: Combinations of networks
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06N 3/096: Transfer learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

This document relates to training of machine learning models such as neural networks. One example method involves providing a machine learning model having one or more layers and associated parameters and performing a pretraining stage on the parameters of the machine learning model to obtain pretrained parameters. The example method also involves performing a tuning stage on the machine learning model by using labeled training samples to tune the pretrained parameters. The tuning stage can include performing noise adjustment of the labeled training examples to obtain noise-adjusted training samples. The tuning stage can also include adjusting the pretrained parameters based at least on the labeled training examples and the noise-adjusted training examples to obtain adapted parameters. The example method can also include outputting a tuned machine learning model having the adapted parameters.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A method performed on a computing device, the method comprising:
accessing training samples for a machine learning model having two or more layers and parameters;
training the machine learning model by:
performing noise adjustment by adding noise to embeddings produced by the machine learning model that represent text of the training samples to obtain noise-adjusted embeddings, and
adjusting the parameters of the machine learning model to obtain a trained machine learning model having adjusted parameters, the adjusting being based at least on a difference between first output distributions determined by the machine learning model using the embeddings representing the text of the training samples and second output distributions determined by the machine learning model using the noise-adjusted embeddings, wherein the first output distributions and the second output distributions are output by the machine learning model during training, and wherein the first output distributions and the second output distributions include different predicted values for a particular training sample; and
outputting the trained machine learning model having the adjusted parameters.

### Claim 22 (depends on 21)

The method of claim 21, wherein the first output distribution includes different first classification likelihoods for different potential classifications of the particular training sample, the second output distribution includes different second classification likelihoods for the different potential classifications of the particular training sample, the different first classification likelihoods being determined using one or more particular embeddings representing the particular training sample and the different second classification likelihoods being determined using one or more particular noise-adjusted embeddings obtained by adding noise to the one or more particular embeddings.

### Claim 23 (depends on 21)

The method of claim 21, the first output distributions and the second output distributions being output by the same machine learning model during training.

### Claim 24 (depends on 21)

The method of claim 21, the first output distributions and the second output distributions being output by a single machine learning model during training.

### Claim 25 (depends on 24)

The method of claim 24, the embeddings being sentence embeddings representing multi-word natural language sentences in the training samples, the noise-adjusted embeddings being obtained by adding noise to the sentence embeddings.

### Claim 26 (depends on 21)

The method of claim 21, wherein the adjusting is performed by computing a loss function, the loss function comprising:
a first term that is proportional to a difference between predictions of the machine learning model determined using the embeddings and values of the training samples, and
a second term that is proportional to the difference between the first output distributions determined by the machine learning model using the embeddings and the second output distributions determined by the machine learning model using the noise-adjusted embeddings.

### Claim 27 (depends on 26)

The method of claim 26, wherein the training comprises multiple training iterations, the method further comprising:
determining a difference between first parameters of a first training iteration of the machine learning model and second parameters of a second training iteration of the machine learning model; and
constraining the adjusting of third parameters of a third training iteration of the machine learning model by imposing a penalty that is calculated based at least on the difference between the first parameters and the second parameters.

### Claim 28 (depends on 21)

The method of claim 21, wherein the training adjusts parameters of an embedding layer of the machine learning model that produces the embeddings and another layer of the machine learning model that outputs the predicted values.

### Claim 29 (depends on 28)

The method of claim 28, wherein the another layer is a task-specific layer.

### Claim 30 (depends on 29)

The method of claim 29, wherein the task-specific layer comprises multiple task-specific layers including at least a single-sentence classification layer, a pairwise text similarity layer, a pairwise text classification layer, and a pairwise ranking layer.

### Claim 31

A system comprising:
a hardware processing unit; and
a storage resource storing computer-readable instructions which, when executed by the hardware processing unit, cause the hardware processing unit to:
receive input data;
process the input data using a machine learning model having two or more layers that have been trained together to obtain a result, the two or more layers having been trained based at least on a difference between first output distributions determined by the machine learning model using embeddings representing text of training samples and second output distributions determined by the machine learning model using noise-adjusted embeddings obtained by adding noise to the embeddings, wherein the first output distributions and the second output distributions are output by the machine learning model during training and include different predicted values for a particular training sample; and
output the result.

### Claim 32 (depends on 31)

The system of claim 31, wherein the input data comprise a query and a document, and the result characterizes similarity of the query to the document.

### Claim 33 (depends on 31)

The system of claim 31, wherein the input data comprise a sentence and the result characterizes a sentiment of the sentence.

### Claim 34 (depends on 33)

The system of claim 33, wherein the result indicates whether the sentence has a positive sentiment or a negative sentiment.

### Claim 35 (depends on 31)

The system of claim 31, wherein the machine learning model includes an embedding layer that produces the embeddings and another layer of the machine learning model that outputs the result, wherein the another layer is a single-sentence classification layer, a pairwise text similarity layer, a pairwise text classification layer, or a pairwise ranking layer.

### Claim 36

A method performed on a computing device, the method comprising:
receiving input data;
processing the input data using a machine learning model having two or more layers that have been trained together to obtain a result, the two or more layers having been trained based at least on a difference between first output distributions determined by the machine learning model using embeddings representing text of training samples and second output distributions determined by the machine learning model using noise-adjusted embeddings obtained by adding noise to the embeddings, wherein the first output distributions and the second output distributions are output by the machine learning model during training and include different predicted values for a particular training sample; and
outputting the result.

### Claim 37 (depends on 36)

The method of claim 36, wherein the machine learning model includes an embedding layer that produces the embeddings and another layer of the machine learning model that outputs the result.

### Claim 38 (depends on 37)

The method of claim 37, wherein the result characterizes sentiments associated with the input data.

### Claim 39 (depends on 38)

The method of claim 38, further comprising:
receiving user input designating a requested sentiment;
filtering multiple statements in the input data to remove individual statements having other sentiments; and
outputting a filtered subset of statements that have the requested sentiment.

### Claim 40 (depends on 39)

The method of claim 39, the multiple statements comprising user reviews of one or more products, the requested sentiment being negative, and the filtering removing positive reviews of the one or more products.
