Machine learning based approach for identification of extremely rare events in high-dimensional space
AssigneeInternational Business Machines Corporation
InventorsSanket Jain, Kumar Apurva, Vikram Yadav
Life of this patent
Estimating the term…
Abstract
Methods and systems for using machine learning to identify extremely rare events in high-dimensional space are disclosed. A method includes: identifying, by a computing device, a plurality of derived attributes using an external data source; selecting, by the computing device, a plurality of key performance indicators from the plurality of derived attributes using a neural network and based on an extremely rare event being modeled; constructing, by the computing device, a linear model using the plurality of key performance indicators; and predicting, by the computing device, occurrences of the extremely rare event using the linear model.
Claims20
A method comprising:
- selecting, by the computing device, a plurality of key performance indicators from a plurality of derived attributes using a neural network and based on an extremely rare event being modeled, each of the plurality of key performance indicators being selected assuming a predetermined minimum number of at least ten events in training data of the neural network for each of the key performance indicators;
- constructing, by the computing device, a linear model using the plurality of key performance indicators including social media data;
- and
- measuring, by the computing device, performance of the linear model on different datasets including out of time (OOT) validation datasets,
- wherein the performance of the linear model is measured by computing accuracy as a measure of a proportional hit rate.
A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:
- identify a plurality of derived attributes;
- select a plurality of key performance indicators from the plurality of derived attributes using a neural network and based on an extremely rare event being modeled;
- construct a linear model using the plurality of key performance indicators;
- predict occurrences of the extremely rare event using the linear model;
- and
- measure performance of the linear model on different datasets, wherein the performance of the linear model is measured by computing accuracy as a measure of a proportional hit rate.
A system comprising:
- a hardware processor, a computer readable memory, and a computer readable storage medium associated with a computing device;
- program instructions to identify a plurality of derived attributes;
- program instructions to select a plurality of key performance indicators from the plurality of derived attributes using a neural network and based on an extremely rare event being modeled, wherein the extremely rare event includes a less than one percent occurrence rate over a predetermined period of time;
- program instructions to integrate social media data indicating reasons for the extremely rare event with the key performance indicators;
- program instructions to construct a linear model using the plurality of key performance indicators;
- and
- program instructions to measure performance of the linear model on different datasets including out of time (OOT) validation datasets,
- wherein the performance of the linear model is measured by computing accuracy as a measure of a proportional hit rate, and
- the program instructions are stored on the computer readable storage medium for execution by the hardware processor via the computer readable memory.
Dependencies are inferred from the claim text.