# US 2021/0342652 A1 · Anomaly detection system using multi-layer support vector machines and method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342652 A1
- Document ID: US20210342652A1
- Status: Published application
- Kind: A1
- Application number: 16/862,867
- Filed: 2020-04-30
- Published: 2021-11-04
- Earliest claimed: 2020-04-30
- Assignees: BAE Systems Information and Electronic Systems Integration Inc.
- Applicants: BAE Systems Information and Electronics Systems Integration Inc.
- Inventors: Martin S. Glassman
- Web page: https://patentgrep.com/patent/US20210342652A1

## CPC

- G06N 20/20: Ensemble learning
- G06F 18/211: Selection of the most significant subset of features
- G06F 18/2113: by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
- G06F 18/213: Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/2411: based on the proximity to a decision surface, e.g. support vector machines
- G06F 18/2433: Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
- G06F 18/251: {of input or preprocessed data}
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/771: Feature selection, e.g. selecting representative features from a multi-dimensional feature space
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A classifier network has at least two distinct sets of refined data, wherein the first two sets of refined data are sets of numbers representing the features values data received from sensors or a manufactured part. Performing, via at least two distinct types of support vector machines using an associated feature selection process for each classifier independently in a first layer, anomaly detection on the manufactured part. Then, using the stored data including refined data of at least two different types of data transforms and performing, via at least a two distinct types of support vector machines in a second layer, an associated feature selection process for each classifier independently. Forming at least four distinct compound classifier types for anomaly detection on the part using the stored data or coefficients. The ensemble of second layer support vector machine outputs compare the results to determine the presence of an anomaly.

## Claims

### Claim 1

A system comprising:
at least two feature types generating transformations of the raw data into arrays or matrices of feature data for each type;
at least two different types of trainable classifiers adapted to have real valued outputs representing statistical confidence in a classification;
feature selection logic associated with each trainable classifier adapted to winnow down the set of feature data from each view that will be used as input to each classifier processing data from that view during system operation;
a first layer of at least two distinct types of classifiers operating on the features output from each feature type in isolation from the processing of data from other feature types;
a feature pooling function that combines the sets of selected features from each feature type into a pooled feature set, wherein the feature pooling uses all features selected for the classifiers in the first layer;
at least two distinct classifier types operating on the pooled feature set;
a second layer of classifiers receiving inputs from outputs of the at least two distinct types of classifiers in the first layer associated with all feature types adapted to form a compound classifier type, wherein the compound classifier includes one type of classifier processing output from another type of classifier or feature set to provide an array of any combination of individual and compound classifier types at the second layer; and
an output function to combine outputs from the layer of classifiers operating on the pooled feature set and the second layer of classifiers in parallel.

### Claim 2 (depends on 1)

The system of claim 1, further comprising:
wherein one of the at least two distinct types of classifier in the first layer operating on the features output is an SVM.

### Claim 3 (depends on 1)

The system of claim 1, further comprising:
a plurality of linear SVMs for recursive feature elimination (RFE) for feature selection adapted to control a feature selection process.

### Claim 4 (depends on 3)

The system of claim 3, further comprising:
RBF SVMs, wherein the plurality of linear SVMs are used as sidecars to control the RFE in conjunction with the RBF SVMs.

### Claim 5 (depends on 3)

The system of claim 3, further comprising:
a grid search classifier hyperparameter optimization wrapper around the RFE adapted to optimize any of the hyperparameters available for each type of classifier with regard to training.

### Claim 6 (depends on 3)

The system of claim 3, further comprising:
a bootstrapped cross validation error rate estimation added to the RFE, where bootstrapped cross validation error rate estimates are obtained with each iteration of the RFE.

### Claim 7 (depends on 6)

The system of claim 6, further comprising:
a margin area measure for the bootstrapped cross validation error rate estimation adapted to establish an optimal operating point with respect to which feature set, over the set of iterations as features are removed, should be the feature set used in operation of the system for each classifier.

### Claim 8 (depends on 1)

The system of claim 1, further comprising sigmoid functions adapted to control the dynamic range of the outputs of each classifier type and normalizing those across the classifiers.

### Claim 9 (depends on 1)

The system of claim 1, further comprising:
a plurality of additional layers of multiple classifier types adapted to form additional compound classifier types to operate as an ensemble with associated adaptive sigmoid functions and a statistical combination of outputs across the plurality of additional layer as the final output of the network.

### Claim 10 (depends on 1)

The system of claim 1, further comprising:
an additional layer including a single classifier adapted to function as an output node for the network that is trained to provide an optimal decision boundary in the feature space composed by the previous layer of classifier outputs.

### Claim 11

A method comprising:
testing a manufactured part for anomalies wherein the testing comprises;
receiving at least one set of feature values data from at least one sensor or the manufactured part;
extracting at least two sets of refined data, wherein the two sets of refined data are sets of numbers representing features values data received from the sensors or manufactured part;
classifying the at least two sets of refined data as a representation of a property or output of the part and storing the at least two sets of refined data as stored data;
performing, via at least a first support vector machine, anomaly detection of the manufactured part using the stored data and performing, via at least a second support vector machine, anomaly detection of the manufactured part using the stored data; and
using at least a second layer including at least a third support vector machine that is the same type of support vector machine as the first support vector machine receiving outputs from the first layer, and at least a fourth support vector machine that is the same type of support vector machine as the second support vector machine receiving outputs from the first layer, connected to provide a parallel set of at least four compound classifier types, the outputs of which are compared or combined to determine the presence of an anomaly.

### Claim 12 (depends on 11)

The method of claim 11, wherein using at least the second layer further comprises:
at least a fifth support vector machine that is the same type of support vector machine as the first support vector machine receiving the pooled feature set as inputs, and at least a sixth support vector machine that is the same type of support vector machine as the second support vector machine receiving the pooled feature set as inputs, connected to provide a parallel set of at least six compound classifier types, the outputs of which are compared or combined to determine the presence of an anomaly.

### Claim 13 (depends on 11)

The method of claim 11 wherein the feature values data represents data from a scan, image, test, or output of the sensor or manufactured part.

### Claim 14 (depends on 11)

The method of claim 11 wherein the method is cloud based.

### Claim 15 (depends on 11)

The method of claim 11 wherein the support vector machines are either radial basis function (RBF) or Linear.

### Claim 16 (depends on 11)

The method of claim 11, wherein performing, based on the minimum specification of a network of support vector machines, anomaly detection on the part using the stored data comprises:
constructing an anomaly classifier based on a decision directed system; and
calculating the presence of an anomaly based on the stored data, wherein the presence of the anomaly comprises at least a difference at which an anomaly occurs.

### Claim 17

An apparatus comprising:
a storage unit, being configured to store at least one of a plurality of feature values data or a plurality of coefficients data in a features or coefficients database, wherein each of the feature values data comprises a plurality of feature values and each of the coefficients values data comprises a plurality of coefficient values;
a non-transitory computer readable storage medium comprising instructions, which when executed by a processor, causes operations configured to:
receive the feature values data of a manufactured part, wherein the feature values data comprises raw feature values data from a scan, image, test, or output of the manufactured part;
extract at least two sets of refined data, wherein the two sets of refined data are numbers representing the feature values data received from the scan, image, test, or output of the manufactured part;
classify the at least two sets of refined data as a representation of a specific mechanical property, chemical property, output of the manufacture part or other similar property of the manufactured part and storing the refined data as stored data; and
a feature database of SVM coefficients or coefficients associated with any similar form of classifier after training coupled to the processor and configured to store, based on classification of the two sets of refined data, the two sets of refined data as the classified raw feature values data, wherein the processor is further configured to perform, based on at least ten support vector machines of at least two distinct types, in an arrangement forming at least two layers of support vector machines of at least two distinct types and an ensemble of compound classifier types formed by combinations of the distinct types of classifiers between a first layer and a second layer processing the at least two sets of refined data, anomaly detection on the manufactured part using the stored data.

### Claim 18 (depends on 17)

The apparatus of claim 17 wherein the support vector machines are either radial basis function (RBF) or Linear.

### Claim 19 (depends on 17)

The apparatus of claim 17, wherein the anomaly detection apparatus further comprises a central database coupled to the microcomputer and the feature database, and wherein the central database is configured to:
parse the features values data to obtain parsed feature values data;
classify the parsed raw sensor data as classified parsed raw sensor data as a representation of a specific mechanical, chemical or other similar property of the part and storing, after classifying the raw parsed sensor data, the classified raw parsed sensor data, wherein the classified raw parsed sensor data corresponds to the at least two sets of refined data, wherein extracting the at least two sets of refined data from the raw sensor data comprises extracting the classified raw parsed sensor data from the raw parsed sensor data.
