Methods and systems for detection of anomalies in digital data streams

a computer system comprising:

• memory;
• a processor in data communication with the memory, the processor configured to determine if an anomaly occurs in a first data stream without reference to a prior analysis of exemplar data streams, by:
• a. receiving a data element included in a first data stream;
• b. determining a pattern for the received data element;
• c. incrementing an occurrence count for a pattern stored in a pattern analysis storage in the memory, if the determined pattern for the received data element matches the stored pattern;
• d. repeating a-c for other data elements in the first data stream;
• e. determining an occurrence count value based on the sum of the occurrence counts of all of the patterns stored in the pattern analysis storage divided by the total number of patterns stored in the pattern analysis storage;
• f. retrieving an occurrence count for a first stored pattern in the pattern analysis storage;
• g. identifying the pattern as an anomaly, if the retrieved occurrence count is below the determined occurrence count value multiplied by a percentage threshold value;
• h. repeating f and g for other patterns stored in the pattern analysis storage;
• and
• i. for each of the data elements in the first data stream, indicating that the data element is anomalous if its associated pattern in the pattern analysis storage is identified as anomalous;
• and
• an output device configured to output an anomaly identifier if the pattern was identified as an anomaly.

A computer system comprising:

• a computer configured to determine if an anomaly occurs in a first data stream without reference to a prior analysis of exemplar data streams, the computer including:
• a. a means for receiving a data element included in the first data stream;
• b. a means for determining a pattern for the received data element;
• c. a means for incrementing an occurrence count for a pattern stored in a pattern analysis storage, if the determined pattern for the received data element matches the stored pattern;
• d. a means for repeating a-c for other data elements in the first data stream;
• e. a means for determining a mean occurrence count value based on the sum of the occurrence counts of all of the patterns stored in the pattern analysis storage over the total number of patterns stored in the pattern analysis storage;
• f. a means for retrieving an occurrence count for a first stored pattern in the pattern analysis storage;
• g. a means for identifying the pattern as an anomaly, if the retrieved occurrence count is below the determined mean occurrence count value multiplied by a percentage threshold value;
• h. a means for repeating f and g for other patterns stored in the pattern analysis storage;
• i. a means for outputting an anomaly identifier if the pattern was identified as an anomaly;
• and
• j. a means for indicating, for each of the data elements in the first data stream, that the data element is anomalous if its associated pattern in the pattern analysis storage is identified as anomalous.