# US 12,007,760 B2 · Anomaly detection and diagnosis in factory automation system using pre-processed time-delay neural network with loss function adaptation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,007,760 B2
- Document ID: US12007760B2
- Status: Granted
- Kind: B2
- Application number: 17/464,901
- Filed: 2021-09-02
- Published: 2023-03-02
- Granted: 2024-06-11
- Earliest claimed: 2021-09-02
- Assignees: Mitsubishi Electric Research Laboratories, Inc.
- Applicants: Mitsubishi Electric Research Laboratories, Inc.
- Inventors: Jianlin Guo; Bryan Liu; Toshiaki Koike Akino; Ye Wang; Kyeong Jin Kim; Kieran Parsons; Philip Orlik
- Web page: https://patentgrep.com/patent/US12007760B2

## CPC

- G05B 23/0283: {Predictive maintenance, e.g. involving the monitoring of a system and, based on the monitoring results, taking decisions on the maintenance schedule of the monitored system; Estimating remaining useful life [RUL] (preventive maintenance, i.e. planning maintenance according to the available resources without monitoring the system G06Q10/06)}
- G05B 19/4184: {characterised by fault tolerance, reliability of production system}
- G05B 23/024: {Quantitative history assessment, e.g. mathematical relationships between available data; Functions therefor; Principal component analysis [PCA]; Partial least square [PLS]; Statistical classifiers, e.g. Bayesian networks, linear regression or correlation analysis; Neural networks}
- G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade}
- G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
- G05B 23/0221: {Preprocessing measurements, e.g. data collection rate adjustment; Standardization of measurements; Time series or signal analysis, e.g. frequency analysis or wavelets; Trustworthiness of measurements; Indexes therefor; Measurements using easily measured parameters to estimate parameters difficult to measure; Virtual sensor creation; De-noising; Sensor fusion; Unconventional preprocessing inherently present in specific fault detection methods like PCA-based methods}
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented pre-processed time-delay autoencoder based anomaly detection method are provided for detecting anomalous states of machines arranged in a factory automation (FA) system or a manufacturing production line. The method includes acquiring source signals from the machines via an interface performing a data pre-processing process for the acquired source signals by normalizing value ranges of the acquired source signals and filtering undesired features from the acquired source signals performing a time-delayed data reform process for the pre-processed source signals based on a time-delay window to generate pre-processed time-delay data submitting pre-processed time-delay testing data to a pre-processed time-delayed autoencoder (Prep-TDAE) neural network, wherein the pre-processed time-delay testing data are collected online while the machines are operated, wherein the Prep-TDAE neural network has been pre-trained by using the pre-processed time-delay training data detecting, if an anomaly state is encountered with respect to the machines, by computing anomaly scores of the pre-processed time-delay testing data, and determining, when the anomaly state is detected, anomaly occurrence time, duration and severity with respect to the anomaly state of each of the machines.

## Claims

### Claim 1

A computer-implemented pre-processed time-delay autoencoder based anomaly detection method for detecting anomalous states of machines arranged in a factory automation (FA) system or a manufacturing production line, comprising steps of:
acquiring source signals from the machines via an interface;
performing a data pre-processing process for the acquired source signals by normalizing value ranges of the acquired source signals and filtering undesired features from the acquired source signals;
performing a time-delayed data reform process for the pre-processed source signals based on a time-delay window to generate pre-processed time-delay data;
submitting pre-processed time-delay testing data to a pre-processed time-delayed autoencoder (Prep-TDAE) neural network, wherein the pre-processed time-delay testing data are collected online while the machines are operated, wherein the Prep-TDAE neural network has been pre-trained by using a pre-processed time-delay training data;
detecting, if an anomaly state is encountered with respect to the machines, by computing anomaly scores of the pre-processed time-delay testing data; and
determining, when the anomaly state is detected, anomaly occurrence time, duration and severity with respect to the anomaly state of each of the machines.

### Claim 2 (depends on 1)

The method of claim 1, wherein the data pre-processing process divides the normalized source signals into normalized analog signals and normalized binary signals.

### Claim 3 (depends on 1)

The method of claim 1, wherein in the determining step, the anomaly state is detected by computing the anomaly score of the normalized signals.

### Claim 4 (depends on 1)

The method of claim 1, wherein an input layer of the Prep-TDAE neural network is configured to accept a binary part of the pre-processed time-delay data via a binary acceptor of the input layer and accept an analog part of the pre-processed time-delay data via an analog acceptor of the input layer.

### Claim 5 (depends on 1)

The method of claim 1, wherein in the step of pre-processing, the acquired signals are divided into binary signals and analog signals, wherein the analog signals are processed by a band-pass filter followed by a high-pass filter, wherein the binary signals are processed by a unit impulse filter.

### Claim 6 (depends on 1)

The method of claim 1, wherein the Prep-TDAE neural network performs loss function adaptation by applying different loss functions to different data types.

### Claim 7 (depends on 6)

The method of claim 6, wherein binary signals are trained with a cross-entropy loss function and analog signals are trained with a squared error loss function.

### Claim 8 (depends on 1)

The method of claim 1, wherein the anomaly occurrence time corresponds to a time index of test data, at which the anomaly score exceeds a predefined threshold.

### Claim 9 (depends on 1)

The method of claim 1, wherein the anomaly duration is the length of the time during which the anomaly score of test data persistently exceeds a predefined threshold for consecutive time indices.

### Claim 10 (depends on 1)

The method of claim 1, wherein the anomaly severity is determined based on a magnitude of the anomaly score.

### Claim 11

A pre-processed time-delay autoencoder based anomaly detection system for detecting anomalous states of machines arranged in a factory automation (FA) system or a manufacturing production line, comprising steps of:
an interface configured to acquire source signals from the machines;
a memory to store computer-executable programs including a data pre-processing algorithm, a time-delayed data reform process algorithm, a pre-processed time-delay autoencoder (Pre-TDAE) neural network, and an anomaly diagnosis algorithm; and
a processor configured to perform steps of:
performing a data pre-processing process algorithm for the acquired source signals by normalizing value ranges of the acquired source signals and filtering undesired features from the acquired source signals;
performing the time-delayed data reform process algorithm for the pre-processed source signals based on a time-delay window to generate pre-processed time-delay data;
submitting pre-processed time-delay testing data to a Prep-TDAE neural network, wherein the pre-processed time-delay testing data are collected online while the machines are operated, wherein the Prep-TDAE neural network has been pre-trained by using pre-processed time-delay training data;
detecting, if an anomaly state is encountered with respect to the machines, by computing anomaly scores of the pre-processed time-delay testing data; and
determining, when the anomaly state is detected, anomaly occurrence time, duration and severity with respect to the anomaly state of each of the machines.

### Claim 12 (depends on 11)

The system of claim 11, wherein the data pre-processing process divides the normalized source signals into normalized analog signals and normalized binary signals.