Method and system for detecting anomalous behavior in stream data

A computing apparatus for detecting an anomalous sequence of events in stream data, the computing apparatus comprising:

• a processor;
• a memory;
• and
• a communication interface coupled to each of the processor and the memory, wherein the processor is configured to:
• receive, via the communication interface, a first set of raw data;
• analyze the first set of raw data in order to determine a first event sequence;
• apply a first Hidden Markov Model (HMM) to the first event sequence in order to generate a first output;
• and
• determine, based on the first output, whether the first event sequence is classifiable as being an anomalous event sequence,
• wherein the first HMM is selected from among a plurality of HMMs that are trained by using a randomized cluster of event sequences and that are configured to generate respective scores based on a result of applying a predetermined algorithm, and wherein the first HMM is selected based on a highest one of the generated respective scores, and
• wherein the processor is further configured to:
• after the first set raw data has been received and before the analysis of the first set of raw data, preprocess the first set of raw data by deleting without augmentation of missing data prior to determination of the first event sequence, wherein the deleting includes performing at least one from among removing at least one null value, removing at least one incomplete action, and eliminating at least one duplication, and wherein the augmentation includes tokenization of data.

A non-transitory computer readable storage medium storing instructions for detecting an anomalous sequence of events in stream data, the storage medium comprising executable code which, when executed by a processor, causes the processor to:

• receive a first set of raw data;
• analyze the first set of raw data in order to determine a first event sequence;
• apply a first Hidden Markov Model (HMM) to the first event sequence in order to generate a first output;
• and
• determine, based on the first output, whether the first event sequence is classifiable as being an anomalous event sequence,
• wherein the first HMM is selected from among a plurality of HMMs that are trained by using a randomized cluster of event sequences and that are configured to generate respective scores based on a result of applying a predetermined algorithm, and wherein the first HMM is selected based on a highest one of the generated respective scores, and
• wherein when executed by the processor, the executable code further causes the processor to:
• after the first set raw data has been received and before the analysis of the first set of raw data, preprocess the first set of raw data by deleting without augmentation of missing data prior to determination of the first event sequence, wherein the deleting includes performing at least one from among removing at least one null value, removing at least one incomplete action, and eliminating at least one duplication, and wherein the augmentation includes tokenization of data.