# US 2025/0384261 A1 · Learning from imperfect data for anomaly detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0384261 A1
- Document ID: US20250384261A1
- Status: Published application
- Kind: A1
- Application number: 18/741,555
- Filed: 2024-06-12
- Published: 2025-12-18
- Earliest claimed: 2024-06-12
- Assignees: Nokia Solutions and Networks Oy
- Applicants: Nokia Solutions and Networks Oy
- Inventors: Mohamed Trabelsi; Ahmet Akyamac
- Web page: https://patentgrep.com/patent/US20250384261A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Apparatus and method of training Machine Learning (ML) models. In an embodiment, the apparatus performs initial training of an anomaly detection model based on training samples of a training dataset over multiple epochs, where the anomaly detection model comprises a variational autoencoder (VAE). For each training sample during an epoch, the initial training comprises inputting an original data sequence of the training sample into the VAE encoder to output a multivariant distribution in latent space, sampling the multivariant distribution to generate multiple latent vectors, inputting the latent vectors into the VAE decoder to output reconstructed data sequences, and computing an estimated sample weight for the training sample. The apparatus identifies, after multiple epochs, corrupted samples from the training dataset based on the estimated sample weights, removes the corrupted samples to generate a filtered training dataset, and performs final training of the anomaly detection model based on the filtered training dataset.

## Claims

### Claim 1

An apparatus, comprising:
at least one processor; and
at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform:
initial training of an anomaly detection model based on training samples of a training dataset over multiple epochs, wherein the anomaly detection model comprises a variational autoencoder;
wherein for each training sample of the training samples during an epoch, the initial training comprises:
inputting an original data sequence corresponding with the training sample into an encoder of the variational autoencoder to output a multivariant distribution in latent space;
sampling the multivariant distribution to generate multiple latent vectors;
inputting the latent vectors into a decoder of the variational autoencoder to output reconstructed data sequences; and
computing an estimated sample weight for the training sample representing an accuracy of the decoder reconstructing the original data sequence from the latent vectors based on the reconstructed data sequences;
identifying, after the multiple epochs, one or more corrupted samples from the training dataset based on estimated sample weights computed for the training samples;
removing the one or more corrupted samples from the training dataset to generate a filtered training dataset; and
performing final training of the anomaly detection model based on the training samples of the filtered training dataset.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the computing the estimated sample weight comprises:
computing reconstruction losses for the latent vectors;
computing a mean of the reconstruction losses for the latent vectors; and
computing the estimated sample weight for the training sample as an inverse of the mean.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the computing the estimated sample weight comprises:
normalizing the estimated sample weight for the training sample within a batch of the training samples.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the identifying one or more corrupted samples from the training dataset comprises:
selecting one or more candidate samples for human feedback based on the estimated sample weights computed for the training samples; and
identifying any of the one or more candidate samples as a corrupted sample when indicated as corrupted based on the human feedback.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the initial training further comprises:
incorporating the human feedback into the anomaly detection model for at least one of the one or more candidate samples.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the incorporating the human feedback comprises:
unlearning any of the one or more candidate samples indicated as a corrupted sample based on the human feedback.

### Claim 7 (depends on 4)

The apparatus of claim 4, wherein the selecting one or more candidate samples comprises:
determining a relative ranking for each of the training samples within a batch of an epoch by sorting the estimated sample weights in decreasing order;
determining ranking distributions for the training samples within the batch over the multiple epochs based on the relative ranking determined for each of the training samples; and
selecting the one or more candidate samples for human feedback based on the ranking distributions.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the identifying the one or more corrupted samples comprises:
identifying at least one of the one or more corrupted samples based on the ranking distributions.

### Claim 9 (depends on 4)

The apparatus of claim 4, wherein, for each training sample of the training samples during the epoch, the initial training further comprises:
augmenting a data sequence of the training sample to generate an augmented data sequence, wherein the augmented data sequence comprises the original data sequence corresponding with the training sample input into the encoder.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein:
the encoder comprises a transformer-based bidirectional encoder; and
the decoder comprises a transformer-based autoregressive decoder.

### Claim 11

A method comprising:
performing initial training of an anomaly detection model based on training samples of a training dataset over multiple epochs, wherein the anomaly detection model comprises a variational autoencoder;
wherein for each training sample of the training samples during an epoch, the initial training comprises:
inputting an original data sequence corresponding with the training sample into an encoder of the variational autoencoder to output a multivariant distribution in latent space;
sampling the multivariant distribution to generate multiple latent vectors;
inputting the latent vectors into a decoder of the variational autoencoder to output reconstructed data sequences; and
computing an estimated sample weight for the training sample representing an accuracy of the decoder reconstructing the original data sequence from the latent vectors based on the reconstructed data sequences;
identifying, after the multiple epochs, one or more corrupted samples from the training dataset based on estimated sample weights computed for the training samples;
removing the one or more corrupted samples from the training dataset to generate a filtered training dataset; and
performing final training of the anomaly detection model based on the training samples of the filtered training dataset.

### Claim 12 (depends on 11)

The method of claim 11, wherein the computing the estimated sample weight comprises:
computing reconstruction losses for the latent vectors;
computing a mean of the reconstruction losses for the latent vectors; and
computing the estimated sample weight for the training sample as an inverse of the mean.

### Claim 13 (depends on 12)

The method of claim 12, wherein the computing the estimated sample weight comprises:
normalizing the estimated sample weight for the training sample within a batch of the training samples.

### Claim 14 (depends on 11)

The method of claim 11, wherein the identifying one or more corrupted samples from the training dataset comprises:
selecting one or more candidate samples for human feedback based on the estimated sample weights computed for the training samples; and
identifying any of the one or more candidate samples as a corrupted sample when indicated as corrupted based on the human feedback.

### Claim 15 (depends on 14)

The method of claim 14, wherein the initial training further comprises:
incorporating the human feedback into the anomaly detection model for at least one of the one or more candidate samples.

### Claim 16 (depends on 15)

The method of claim 15, wherein the incorporating the human feedback comprises:
unlearning any of the one or more candidate samples indicated as a corrupted sample based on the human feedback.

### Claim 17 (depends on 14)

The method of claim 14, wherein the selecting one or more candidate samples comprises:
determining a relative ranking for each of the training samples within a batch of an epoch by sorting the estimated sample weights in decreasing order;
determining ranking distributions for the training samples within the batch over the multiple epochs based on the relative ranking determined for each of the training samples; and
selecting the one or more candidate samples for human feedback based on the ranking distributions.

### Claim 18 (depends on 17)

The method of claim 17, wherein the identifying the one or more corrupted samples comprises:
identifying at least one of the one or more corrupted samples based on the ranking distributions.

### Claim 19 (depends on 14)

The method of claim 14, wherein, for each training sample of the training samples during the epoch, the initial training further comprises:
augmenting a data sequence of the training sample to generate an augmented data sequence, wherein the augmented data sequence comprises the original data sequence corresponding with the training sample input into the encoder.

### Claim 20

A computer readable medium embodying programmed instructions which, when executed by a processor, are operable for performing a method comprising:
performing initial training of an anomaly detection model based on training samples of a training dataset over multiple epochs, wherein the anomaly detection model comprises a variational autoencoder;
wherein for each training sample of the training samples during an epoch, the initial training comprises:
inputting an original data sequence corresponding with the training sample into an encoder of the variational autoencoder to output a multivariant distribution in latent space;
sampling the multivariant distribution to generate multiple latent vectors;
inputting the latent vectors into a decoder of the variational autoencoder to output reconstructed data sequences; and
computing an estimated sample weight for the training sample representing an accuracy of the decoder reconstructing the original data sequence from the latent vectors based on the reconstructed data sequences;
identifying, after the multiple epochs, one or more corrupted samples from the training dataset based on estimated sample weights computed for the training samples;
removing the one or more corrupted samples from the training dataset to generate a filtered training dataset; and
performing final training of the anomaly detection model based on the training samples of the filtered training dataset.
