US 2026/0028047 A1 · Pre-Training Machine Learning Models with Contrastive Learning

Abstract

Provided are methods for pre-training machine learning models with contrastive learning. Some methods described also include generating, with at least one processor, a perturbed dataset from a real dataset. The method includes pre-training, with the at least one processor, at least one component of a machine leaning model to perform an alternative task, wherein the machine learning model performs a primary task. The method also includes inserting, with the at least one processor, the pre-trained at least one component into the machine learning model that performs the primary task. Additionally, the method includes training, with the at least one processor, the machine learning model comprising the pre-trained at least one component to perform the primary task. Systems and computer program products are also provided.

Claims20

Dependent claims

A method, comprising:

generating, with at least one processor, a perturbed dataset from a real dataset;
pre-training, with the at least one processor, at least one component of a machine leaning model to perform an alternative task, wherein the machine learning model performs a primary task;
inserting, with the at least one processor, the pre-trained at least one component into the machine learning model that performs the primary task;
and
training, with the at least one processor, the machine learning model comprising the pre-trained at least one component to perform the primary task.

2 more elements

Dependencies are inferred from the claim text.

Pre-Training Machine Learning Models with Contrastive Learning

Life of this patent

Pre-Training Machine Learning Models with Contrastive Learning

Life of this patent

Abstract

Claims20