# US 12,217,186 B2 · Method and apparatus for semi-supervised learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,186 B2
- Document ID: US12217186B2
- Status: Granted
- Kind: B2
- Application number: 17/436,927
- Filed: 2021-05-25
- Published: 2023-06-08
- Granted: 2025-02-04
- Earliest claimed: 2020-05-28
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Ivana Balazevic; Carl Allen; Timothy Hospedales
- Web page: https://patentgrep.com/patent/US12217186B2

## CPC

- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7753: {Incorporation of unlabelled data, e.g. multiple instance learning [MIL]}
- G06V 10/7784: {based on feedback from supervisors}
- G06V 10/82: using neural networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Provided is a computer-implemented method for training a machine learning (ML) model using labelled and unlabelled data, the method comprising obtaining a set or training data comprising a set of labelled data items and a set of unlabelled data items, training a loss module of the ML model using labels in the set of labelled data items, to generate a trained loss module capable of estimating a likelihood of a label for a data item, and training a task module of the ML model using the loss module, the set of labelled data items, and the set of unlabelled data items, to generate a trained task module capable of making a prediction of a label for input data.

## Claims

### Claim 1

A computer-implemented method for training a machine learning (ML) model using labelled and unlabelled data, the method comprising:
obtaining a set of training data comprising a set of labelled data items and a set of unlabelled data items;
training a loss module of the ML model using labels in the set of labelled data items, to generate a trained loss module capable of estimating a likelihood of a label for a data item; and
training a task module of the ML model using the loss module, the set of labelled data items, and the set of unlabelled data items, to generate a trained task module capable of making a prediction of a label for input data,
wherein the training of the task module comprises:
inputting a data item in the set of labelled data items into the task module and outputting, from the task module, a predicted label for the data item in the set of labelled data items;
comparing, using a supervised loss module, the output predicted label for the data item in the set of labelled data items with an actual label of the data item in the set of labelled data items, and determining a supervised loss function based on a result of the comparing;
inputting a data item in the set of unlabelled data items into the task module and outputting, from the task module, a predicted label for the data item in the set of unlabelled data items;
determining an unsupervised loss function using the trained loss module and the output predicted label, output from the task module, for the data item in the set of unlabelled data items, wherein the unsupervised loss function is defined by a likelihood of the predicted label for the data item in the set of unlabelled data items;
calculating a total loss function based on a sum of the supervised loss function and the unsupervised loss function; and
using the total loss function and backpropagation to train the task module by minimizing the sum of the supervised loss function and the unsupervised loss function.

### Claim 2 (depends on 1)

The method as claimed in claim 1, wherein the loss module of the ML model comprises a generative neural network or a normalising flow neural network.

### Claim 3 (depends on 1)

The method as claimed in claim 1, wherein the training of the loss module comprises:
determining a loss function representing a likelihood of a label in the set of labelled data items; and
using the determined loss function and backpropagation to minimize the loss function.

### Claim 4

A computer-implemented method for training a machine learning (ML) model using labelled and unlabelled data, the method comprising:
obtaining a set of training data comprising a set of labelled data items and a set of unlabelled data items;
training a loss module of the ML model using labels in the set of labelled data items, to generate a trained loss module capable of estimating a likelihood of a label for a data item, wherein the task module outputs a predicted label for a data item in the set of unlabelled data items, and the loss module is trained further using the predicted label for the data item in the set of unlabelled data items;
training a task module of the ML model using the trained loss module, the set of labelled data items, and the set of unlabelled data items, to generate a trained task module capable of making a prediction of a label for input data, wherein the task module outputs both the predicted label for the data item in the set of unlabelled data items and a predicted label for a data item in the set of labelled data items; and
wherein the training of the loss module and the training of the task module comprises alternating between a step to train the loss module and a step to train the task module for a fixed number of iterations or until convergence.

### Claim 5 (depends on 4)

The method as claimed in claim 4, wherein the step to train the loss module comprises:
obtaining, from the task module, the predicted label for the data item in the set of unlabelled data items;
obtaining an actual label for a data item in the set of labelled data items;
determining, using the predicted label and the actual label, a loss function; and
using the loss function and backpropagation to train the loss module to minimise the loss function.

### Claim 6 (depends on 4)

The method as claimed in claim 4, wherein the step to train the task module comprises:
inputting the data item in the set of labelled data items into the task module and obtaining the predicted label for the data item in the set of labelled data items;
comparing, using a supervised loss module, the output predicted label for the data item in the set of labelled data items with an actual label of the data item in the set of labelled data items, and determining a supervised loss function;
inputting a data item in the set of unlabelled data items into the task module and obtaining the predicted label for the data item in the set of unlabelled data items;
determining an unsupervised loss function using the loss module and the output predicted label for the data item in the set of unlabelled data items;
calculating a total loss function based on the supervised loss function and the unsupervised loss function; and
using the total loss function and backpropagation to train the task module.

### Claim 7 (depends on 1)

The method as claimed in claim 1, wherein the trained task module is
trained to perform a semantic image segmentation and wherein the prediction of the label for the input data comprises a prediction of a semantic image segmentation for an image in the input data.

### Claim 8 (depends on 1)

The method as claimed in claim 1, wherein the trained task module is trained to analyse images of food, and wherein the prediction of the label for the input data comprises a prediction of at least one food-based descriptor for a food image in the input data.

### Claim 9 (depends on 1)

The method as claimed in claim 1, wherein the trained task module is trained to perform gesture analysis or pose estimation, and wherein the prediction of the label for the input data comprises a prediction of at least one gesture or pose shown in an image in the input data.

### Claim 10 (depends on 1)

A non-transitory computer-readable recording medium having recorded thereon code which, when implemented on a processor, causes the processor to carry out the method of claim 1.

### Claim 11

An apparatus for training a machine learning (ML) model using labelled and unlabelled data, the apparatus comprising:
at least one memory; and
at least one processor coupled to the at least one memory and configured to:
obtain a set of training data comprising a set of labelled data items and a set of unlabelled data items,
train a loss module of the ML model using labels in the set of labelled data items, to generate a trained loss module capable of estimating a likelihood of a label for a data item, and
train a task module of the ML model using the loss module, the set of labelled data items, and the set of unlabelled data items, to generate a trained task module capable of making a prediction of a label for input data,
wherein the at least one processor is configured to train the task module by:
inputting a data item in the set of labelled data items into the task module and outputting, from the task module, a predicted label for the data item in the set of labelled data items;
comparing, using a supervised loss module, the output predicted label for the data item in the set of labelled data items with an actual label of the data item in the set of labelled data items, and determining a supervised loss function based on a result of the comparing;
inputting a data item in the set of unlabelled data items into the task module and outputting, from the task module, a predicted label for the data item in the set of unlabelled data items;
determining an unsupervised loss function using the trained loss module and the output predicted label, output from the task module, for the data item in the set of unlabelled data items, wherein the unsupervised loss function is defined by a likelihood of the predicted label for the data item in the set of unlabelled data items;
calculating a total loss function based on a sum of the supervised loss function and the unsupervised loss function; and
using the total loss function and backpropagation to train the task module by minimizing the sum of the supervised loss function and the unsupervised loss function.

### Claim 12 (depends on 11)

The apparatus as claimed in claim 11 further comprising storage configured to store data items obtained by a user of the apparatus, and wherein the set of unlabelled data items comprises the data items in the storage.

### Claim 13 (depends on 12)

The apparatus as claimed in claim 12, wherein the loss module of the ML model comprises a normalising flow neural network.

### Claim 14 (depends on 12)

The apparatus as claimed in claim 12, wherein the at least one processor is configured to train the loss module by:
determining a loss function representing a likelihood of a label in the set of labelled data items; and
using the determined loss function and backpropagation to minimise the loss function.