Back to results
US 12,613,761 B1Granted
Data science anomaly detection
AssigneeBlackRock Finance, Inc.
InventorsAmit Panalkar, Rishab Chauhan, Pranay Sharma, Earvin Laxamana
Life of this patent
Estimating the term…
Abstract
A system, method, and a computer program product for determining anomalies in a dataset are provided. The dataset, which stores data structures of parameters, is received at an anomaly detection framework. The anomaly detection framework selects multiple models, including time series models, historical models, artificial intelligence models and isolation forest models to analyze the parameters in the dataset and determine anomalies in the data structures.
Claims20
A system, comprising:
- a memory configured to store a plurality of anomaly detection models;
- and
- a processor coupled to the memory and configured to perform operations, the operations comprising:
- receiving data structures in a dataset transmitted by a computing network from a plurality of data sources coupled to the computer network, wherein the data structures include parameters storing transaction data generated by the plurality of data sources;
- determining a first anomaly detection output for a data structure in the data structures using a first anomaly detection model in the plurality of anomaly detection models by:
- comparing first parameters associated with the data structure to other first parameters associated with other data structures;
- generating a parameter score for each first parameter in the first parameters based on the comparison;
- aggregating parameter scores for the first parameters into an aggregated parameter score;
- and
- generating the first anomaly detection output based on the aggregated parameter score;
- determining a second anomaly detection output for the data structure using a second anomaly detection model in the plurality of anomaly detection models by:
- providing a random forest of trees generated using minimum and maximum values of second parameters in the dataset;
- isolating second parameters in the data structure using the random forest of trees by passing the second parameters in the data structure through trees in the random forest of trees;
- determining path lengths of the second parameters of the data structure through the trees;
- and
- generating the second anomaly detection output based on the path lengths;
- determining a third anomaly detection output for the data structure using a third anomaly detection model in the plurality of anomaly detection models by:
- generating third anomaly detection outputs over a historical time period;
- and
- determining the third anomaly detection output based on a number of times second anomaly detection outputs indicated an anomaly in the data structure over the historical time period;
- and
- determining a fourth anomaly detection output for the data structure in the dataset using a fourth anomaly detection model in the plurality of anomaly detection models by:
- identifying fourth parameters from parameters associated with the dataset;
- generating distributions of the fourth parameters over a second historical time period;
- determining expected range of values in the distributions using the fourth parameters;
- and
- generating the fourth anomaly detection output based on fourth parameters in the data structure and the expected range of values;
- determining that the data structure includes the anomaly based on the first anomaly detection output, the second anomaly detection output, the third anomaly detection output, and the fourth anomaly detection output;
- and
- generating an alert for the data structure based on the anomaly;
- and
- transmitting the alert to a computing device, wherein the alert, upon receipt at the computing device, activates an icon on a display screen of the computing device indicating that the alert with the anomaly has been received at the computing device.
29 more elements
A method, comprising:
- receiving a dataset including data structures storing parameters over a computer network, wherein a ground truth or expected parameter values for the parameters are unknown;
- selecting, based on an available memory of a computing device, a first anomaly detection model, a second anomaly detection model, a third anomaly detection model, and a fourth anomaly detection model from a plurality of anomaly detection models;
- determining, using at least one processor and the available memory, a first anomaly detection output for a data structure in the dataset using the first anomaly detection model in the plurality of anomaly detection models and first parameters in the data structure;
- determining, using the at least one processor and the available memory, a second anomaly detection output for the data structure using the second anomaly detection model in the plurality of anomaly detection models and second parameters in the data structure;
- determining, using the at least one processor and the available memory, a third anomaly detection output using the third anomaly detection model in the plurality of anomaly detection models and third parameters in the data structure;
- determining, using the at least one processor and the available memory, a fourth anomaly detection output using the fourth anomaly detection model in the plurality of anomaly detection models and fourth parameters in the data structure;
- and
- determining that the data structure is an anomalous data structure based on the first anomaly detection output, the second anomaly detection output, the third anomaly detection output, and the fourth anomaly detection output,
- wherein determining using the first anomaly detection model, the second anomaly detection model, the third anomaly detection model, and the fourth anomaly detection model is performed in parallel.
6 more elements
A non-transitory computer readable medium storing instruction thereon, that when executed by a processor cause the processor to perform operations for detecting an anomaly in at least one data structure of a computer network, the operations comprising:
- receiving a dataset including data structures over the computer network, the data structures storing parameters wherein a ground truth or expected parameter values for the parameters are unknown;
- dividing the data structures in the dataset into a plurality of cohorts using a subset of the parameters in the data structures, wherein at least one parameter in the subset of parameters is an Internet Protocol address;
- selecting a plurality of anomaly detection models;
- processing, using the plurality of anomaly detection models executing in parallel, data structures in each cohort in the plurality of cohorts, wherein each anomaly detection model generates anomaly detection scores for the data structures of each cohort in the plurality of cohorts;
- aggregating an anomaly detection score in the anomaly detection scores for each data structure;
- generating an alert for the at least one data structure based on the corresponding aggregated anomaly detection score;
- and
- transmitting the alert, wherein the alert, upon receipt at a computing device, activates an icon on a display screen of the computing device indicating that the alert with the anomaly detection score has been received at the computing device.
5 more elements
Dependencies are inferred from the claim text.