Back to results
US 2021/0144167 A1Published application
Apparatus and method for detecting an anomaly in a dataset and computer program product therefor
AssigneeHuawei Technologies Co., Ltd.
InventorsValery Nikolaevich GLUKHOV, Liang Zhang, Jiyu Pan
Life of this patent
Estimating the term…
Abstract
Apparatus and methods for detecting an anomaly in a dataset by using two or more anomaly detection algorithms, as well as to corresponding computer program products, are described. The results obtained by using the two or more anomaly detection algorithms are combined in accordance with a certain rule of combination, thereby providing an improved accuracy of anomaly detection.
Claims20
An apparatus for detecting an anomaly in a dataset, the apparatus comprising:
- at least one processor;
- and
- a storage coupled to the at least one processor and storing executable instructions which, when executed by the at least one processor, cause the at least one processor to:
- receive the dataset comprising multiple data items among which at least one data item is anomalous,
- process the data items in the data set by each of at least two of a plurality of anomaly detection algorithms to:
- calculate an anomaly score for each of the data items,
- based on the anomaly scores, obtain a partial ranking of the data items, the partial ranking causing the data items to be divided into subsets each corresponding to a different interval of intermediate ranks,
- based on the partial ranking, select a probabilistic model describing the intermediate ranks of the data items in each subset, and
- based on the probabilistic model, assign a degree of belief to the intermediate rank of each of the data items in each subset,
- obtain a total degree of belief for the intermediate rank of each of the data items by combining the degrees of belief obtained, for intermediate ranks corresponding to each of the data items, from the at least two anomaly detection algorithms in accordance with a predefined combination rule,
- convert the total degrees of belief for the intermediate ranks of the data items to a probability distribution function describing expected ranks of the data items,
- sort the data items according to the expected ranks of the data items, and
- find, among the sorted data items, the at least one anomalous data item.
10 more elements
A method for detecting an anomaly in a dataset, the method comprising:
- receiving the dataset comprising multiple data items among which at least one data item is anomalous,
- processing the data items in the data set by each of at least two of a plurality of anomaly detection algorithms by:
- calculating an anomaly score for each of the data items,
- based on the anomaly scores, obtaining a partial ranking of the data items, the partial ranking causing the data items to be divided into subsets each corresponding to a different interval of intermediate ranks,
- based on the partial ranking, selecting a probabilistic model describing the intermediate ranks of the data items in each subset, and
- based on the probabilistic model, assigning a degree of belief to the intermediate rank of each of the data items in each subset,
- obtaining a total degree of belief for the intermediate rank of each of the data items by combining the degrees of belief obtained, for intermediate ranks corresponding to each of the data items, from the at least two anomaly detection algorithms in accordance with a predefined combination rule,
- converting the total degrees of belief for the intermediate ranks of the data items to a probability distribution function describing expected ranks of the data items,
- sorting the data items according to the expected ranks of the data items, and
- finding, among the sorted data items, the at least one anomalous data item.
7 more elements
A computer program product comprising a computer-readable storage medium storing a computer program, the computer program, when executed by at least one processor, causing the at least one processor to perform operations, comprising:
- receiving the dataset comprising multiple data items among which at least one data item is anomalous,
- processing the data items in the data set by each of at least two of a plurality of anomaly detection algorithms by:
- calculating an anomaly score for each of the data items,
- based on the anomaly scores, obtaining a partial ranking of the data items, the partial ranking causing the data items to be divided into subsets each corresponding to a different interval of intermediate ranks,
- based on the partial ranking, selecting a probabilistic model describing the intermediate ranks of the data items in each subset, and
- based on the probabilistic model, assigning a degree of belief to the intermediate rank of each of the data items in each subset,
- obtaining a total degree of belief for the intermediate rank of each of the data items by combining the degrees of belief obtained, for intermediate ranks corresponding to each of the data items, from the at least two anomaly detection algorithms in accordance with a predefined combination rule,
- converting the total degrees of belief for the intermediate ranks of the data items to a probability distribution function describing expected ranks of the data items,
- sorting the data items according to the expected ranks of the data items, and
- finding, among the sorted data items, the at least one anomalous data item.
7 more elements
Dependencies are inferred from the claim text.