# US 2019/0244337 A1 · Unsupervised image-based anomaly detection using multi-scale context-dependent deep autoencoding gaussian mixture model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244337 A1
- Document ID: US20190244337A1
- Status: Published application
- Kind: A1
- Application number: 16/248,955
- Filed: 2019-01-16
- Published: 2019-08-08
- Earliest claimed: 2018-02-05
- Assignees: NEC Laboratories America, Inc.
- Applicants: NEC Laboratories America, Inc.
- Inventors: Alexandru Niculescu-Mizil; Renqiang Min; Eric Cosatto; Farley Lai; Hans Peter Graf; Xavier Fontaine
- Web page: https://patentgrep.com/patent/US20190244337A1

## CPC

- G06T 7/0004: {Industrial image inspection}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/2414: {Smoothing the distance, e.g. radial basis function networks [RBFN]}
- G06F 18/2433: Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
- G06T 3/403: Edge-driven scaling; Edge-based scaling
- G06T 7/001: {using an image reference approach}
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G08B 29/186: {Fuzzy logic; neural networks}
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06V 2201/06: Recognition of objects for industrial automation
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS

## Abstract

A false alarm reduction system is provided that includes a processor cropping each input image at randomly chosen positions to form cropped images of a same size at different scales in different contexts. The system further includes a CONDA-GMM, having a first and a second conditional deep autoencoder for respectively (i) taking each cropped image without a respective center block as input for measuring a discrepancy between a reconstructed and a target center block, and (ii) taking an entirety of cropped images with the target center block. The CONDA-GMM constructs density estimates based on reconstruction error features and low-dimensional embedding representations derived from image encodings. The processor determines an anomaly existence based on a prediction of a likelihood of the anomaly existing in a framework of a CGMM, given the context being a representation of the cropped image with the center block removed and having a discrepancy above a threshold.

## Claims

### Claim 1

A false alarm reduction system for reducing false alarms in an automatic defect detection system, the false alarm reduction system comprising:
a processor for performing a cropping operation on each of input images at randomly chosen positions to form a set of cropped images of a same size at different scales in different contexts; and
a CONtext-conditional Deep Autoencoding Gaussian Mixture Model (CONDA-GMM), having a first and a second conditional deep autoencoder for respectively (i) taking each of the cropped images without a respective center block as input for measuring a discrepancy between a reconstructed center block and the target center block, and (ii) taking an entirety of the cropped images with the target center block, the CONDA-GMM constructing density estimates based on both reconstruction error features and low-dimensional embedding representations derived from encodings of the cropped images,
wherein the processor determines an existence of an anomaly based on a prediction of a likelihood of the anomaly existing in a framework of a context-dependent Conditional Gaussian Mixture Model (CGMM), given the context being a representation of the cropped image with the center block removed and having a discrepancy above a threshold amount.

### Claim 2 (depends on 1)

The false alarm reduction system of claim 1, wherein the CONDA-GMM comprises a context-dependent deep autoencoding compression network and a context-dependent density estimation network.

### Claim 3 (depends on 1)

The false alarm reduction system of claim 1, wherein each of the input images is cropped at multiple different randomly chosen positions.

### Claim 4 (depends on 1)

The false alarm reduction system of claim 1, wherein a cropping-specific contextual vector is formed using image features of the cropped images.

### Claim 5 (depends on 4)

The false alarm reduction system of claim 4, wherein the image features comprise Scale Invariant Feature Transformation (SIFT) features.

### Claim 6 (depends on 4)

The false alarm reduction system of claim 4, wherein the image features forming the cropping-specific contextual vector comprise edges of the cropped images, wherein the cropped edges have been cropped to be without a center region.

### Claim 7 (depends on 1)

The false alarm reduction system of claim 1, wherein the CONDA-GMM includes a first and a second conditional deep autoencoder for respectively taking each of the cropped images without a respective center block as input for measuring a discrepancy between a reconstructed center block and the target center block, and taking an entirety of the cropped images with the target center block.

### Claim 8 (depends on 1)

The false alarm reduction system of claim 1, wherein the likelihood is based on an energy level.

### Claim 9 (depends on 1)

The false alarm reduction system of claim 1, wherein both reconstruction errors and log-likelihood are used for the anomaly detection.

### Claim 10 (depends on 1)

The false alarm reduction system of claim 1, further comprising a Multi-Layer Perceptron, predicting an input-specific mixture coefficient conditioned on the representation.

### Claim 11

A false alarm reduction method for reducing false alarms in an automatic defect detection system, the false alarm reduction method comprising:
performing, by a processor, a cropping operation on each of input images at randomly chosen positions to form a set of cropped images of a same size at different scales in different contexts;
encoding, by a first conditional deep autoencoder of a CONtext-conditional Deep Autoencoding Gaussian Mixture Model (CONDA-GMM), each of the cropped images without a respective center block as input for measuring a discrepancy between a reconstructed center block and the target center block, and
encoding, by a second conditional deep autoencoder of the CONDA-GMM, an entirety of the cropped images with the target center block;
constructing, by the CONDA-GMM, density estimates based on both reconstruction error features and low-dimensional embedding representations derived from encodings of the cropped images; and
determining, by the processor, an existence of an anomaly based on a prediction of a likelihood of the anomaly existing in a framework of a context-dependent Conditional Gaussian Mixture Model (CGMM), given the context being a representation of the cropped image with the center block removed and having a discrepancy above a threshold amount.

### Claim 12 (depends on 11)

The false alarm reduction method of claim 11, wherein the CONDA-GMM comprises a context-dependent deep autoencoding compression network and a context-dependent density estimation network.

### Claim 13 (depends on 11)

The false alarm reduction method of claim 11, wherein each of the input images is cropped at multiple different randomly chosen positions.

### Claim 14 (depends on 11)

The false alarm reduction method of claim 11, further comprising forming a cropping-specific contextual vector using image features of the cropped images.

### Claim 15 (depends on 14)

The false alarm reduction method of claim 14, wherein the image features comprise Scale Invariant Feature Transformation (SIFT) features.

### Claim 16 (depends on 14)

The false alarm reduction method of claim 14, wherein the image features forming the cropping-specific contextual vector comprise edges of the cropped images, wherein the cropped edges have been cropped to be without a center region.

### Claim 17 (depends on 11)

The false alarm reduction method of claim 11, wherein the CONDA-GMM includes a first and a second conditional deep autoencoder for respectively taking each of the cropped images without a respective center block as input for measuring a discrepancy between a reconstructed center block and the target center block, and taking an entirety of the cropped images with the target center block.

### Claim 18 (depends on 11)

The false alarm reduction method of claim 11, wherein the likelihood is based on an energy level.

### Claim 19 (depends on 11)

The false alarm reduction method of claim 11, wherein both reconstruction errors and log-likelihood are used for the anomaly detection.

### Claim 20 (depends on 11)

The false alarm reduction method of claim 11, further comprising predicting, by a Multi-Layer Perceptron, an input-specific mixture coefficient conditioned on the representation.