System and method for real time distributed adaptive cube aggregation, heuristics based hierarchical clustering and anomaly detection framework for high volume streaming datasets

A computer system for aggregating an incoming data stream, the computer system comprising:

• a memory device for storing a plurality of data from the incoming data stream;
• a non-transitory computer readable medium;
• a network connection capable of receiving the incoming data stream;
• and
• a processor in communication with said memory device, said non-transitory computer readable medium, and said network connection, said processor configured to perform a deep packet inspection (DPI) and further configured to:
• establish the incoming data stream to the computing system;
• parse on-the-fly said DPI data on a per-subscriber basis based into a plurality of dimensions, said plurality of dimensions correspond to a plurality of subscriber session attributes and a plurality of values corresponding to said plurality of subscriber session attributes;
• separate on-the-fly a plurality of volatile datasets of said plurality of subscriber session attributes, a plurality of semi-volatile datasets of said plurality of subscriber session attributes, and a plurality of non-volatile datasets of said plurality of subscriber session attributes into a plurality of distinct memory bins for each plurality of datasets;
• expand an at least one of said plurality of distinct memory bins when said at least one of said plurality of distinct memory bins is consumed;
• generate an in-memory hypercube having a plurality of hypercube dimensions each corresponding to said plurality of dimensions;
• numerically translate an input query into an access query for said in-memory hypercube, said input query comprising an at least one subscriber attribute;
• and
• return a list of subscribers in response to said input query;
• wherein said plurality of subscriber session attributes comprise a download volume, an upload volume, a traffic type, a session duration, a geolocation, and a quality of service.