# US 2026/0028047 A1 · Pre-Training Machine Learning Models with Contrastive Learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0028047 A1
- Document ID: US20260028047A1
- Status: Published application
- Kind: A1
- Application number: 18/286,670
- Filed: 2023-04-14
- Published: 2026-01-29
- Earliest claimed: 2023-04-14
- Assignees: Motional AD LLC
- Applicants: Motional AD LLC
- Inventors: Nanxiang Li; Bo Yang
- Web page: https://patentgrep.com/patent/US20260028047A1

## CPC

- B60W 60/0027: {using trajectory prediction for other traffic participants}
- G06N 20/00: Machine learning
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/096: Transfer learning
- G06V 10/764: using classification, e.g. of video objects
- G06N 3/00: Computing arrangements based on biological models
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Provided are methods for pre-training machine learning models with contrastive learning. Some methods described also include generating, with at least one processor, a perturbed dataset from a real dataset. The method includes pre-training, with the at least one processor, at least one component of a machine leaning model to perform an alternative task, wherein the machine learning model performs a primary task. The method also includes inserting, with the at least one processor, the pre-trained at least one component into the machine learning model that performs the primary task. Additionally, the method includes training, with the at least one processor, the machine learning model comprising the pre-trained at least one component to perform the primary task. Systems and computer program products are also provided.

## Claims

### Claim 1

A method, comprising:
generating, with at least one processor, a perturbed dataset from a real dataset;
pre-training, with the at least one processor, at least one component of a machine leaning model to perform an alternative task, wherein the machine learning model performs a primary task;
inserting, with the at least one processor, the pre-trained at least one component into the machine learning model that performs the primary task; and
training, with the at least one processor, the machine learning model comprising the pre-trained at least one component to perform the primary task.

### Claim 2 (depends on 1)

The method of claim 1, wherein generating the perturbed dataset comprises modifying a past trajectory of an autonomous vehicle, a future trajectory of the autonomous vehicle, trajectories of actors, adding or removing lanes, adding or removing actors, or any combinations thereof.

### Claim 3 (depends on 1)

The method of claim 1, wherein the real dataset is a real world driving log.

### Claim 4 (depends on 1)

The method of claim 1, wherein the pre-trained at least one component is stored as an intermediate model that is iteratively updated when new data is available.

### Claim 5 (depends on 1)

The method of claim 1, wherein the at least one component captures interactions between actor features and map features.

### Claim 6 (depends on 1)

The method of claim 1, wherein the alternative task is a classification task.

### Claim 7 (depends on 1)

The method of claim 1, wherein the primary task is a planning task or a prediction task.

### Claim 8

A system, comprising:
at least one processor, and
at least one non-transitory storage media storing instructions that, when executed by the at least one processor, cause the at least one processor to:
generate a perturbed dataset from a real dataset;
pre-train at least one component of a machine leaning model to perform an alternative task, wherein the machine learning model performs a primary task;
insert the pre-trained at least one component into the machine learning model that performs the primary task; and
train the machine learning model comprising the pre-trained at least one component to perform the primary task.

### Claim 9 (depends on 1)

The system of claim 1, wherein generating the perturbed dataset comprises modifying a past trajectory of an autonomous vehicle, a future trajectory of the autonomous vehicle, trajectories of actors, adding or removing lanes, adding or removing actors, or any combinations thereof.

### Claim 10 (depends on 1)

The system of claim 1, wherein the real dataset is a real world driving log.

### Claim 11 (depends on 1)

The system of claim 1, wherein the pre-trained at least one component is stored as an intermediate model that is iteratively updated when new data is available.

### Claim 12 (depends on 1)

The system of claim 1, wherein the at least one component captures interactions between actor features and map features.

### Claim 13 (depends on 1)

The system of claim 1, wherein the alternative task is a classification task.

### Claim 14 (depends on 1)

The system of claim 1, wherein the primary task is a planning task or a prediction task.

### Claim 15

At least one non-transitory storage media storing instructions that, when executed by at least one processor, cause the at least one processor to:
generate a perturbed dataset from a real dataset;
pre-train at least one component of a machine leaning model to perform an alternative task, wherein the machine learning model performs a primary task;
insert the pre-trained at least one component into the machine learning model that performs the primary task; and
train the machine learning model comprising the pre-trained at least one component to perform the primary task.

### Claim 16 (depends on 1)

The at least one non-transitory storage media of claim 1, wherein generating the perturbed dataset comprises modifying a past trajectory of an autonomous vehicle, a future trajectory of the autonomous vehicle, trajectories of actors, adding or removing lanes, adding or removing actors, or any combinations thereof.

### Claim 17 (depends on 1)

The at least one non-transitory storage media of claim 1, wherein the real dataset is a real world driving log.

### Claim 18 (depends on 1)

The at least one non-transitory storage media of claim 1, wherein the pre-trained at least one component is stored as an intermediate model that is iteratively updated when new data is available.

### Claim 19 (depends on 1)

The at least one non-transitory storage media of claim 1, wherein the at least one component captures interactions between actor features and map features.

### Claim 20 (depends on 1)

The at least one non-transitory storage media of claim 1, wherein the alternative task is a classification task.
