System and method for stream data type identification using machine learning

A computer system comprising a hardware memory, wherein the computer system is configured to execute software instructions stored on nontransitory machine-readable storage media that:

• identify a data type of digital data received in varying formats by adapting statistical analysis techniques based on whether received digital data comprises a discrete file with determinable boundaries or a continuous data stream without predetermined endpoints;
• when the digital data comprises a discrete file:
• segment the entire file into groups of bytes and generate a statistical file-print comprising a plurality of statistical characteristics of a distribution of the groups of bytes across the entire file;
• when the digital data comprises a continuous data stream:
• maintain rolling buffers of configurable size to capture streaming data segments;
• generate incremental statistical file-prints from buffered segments using sliding window analysis, each incremental file-print comprising statistical characteristics of data within a temporal window;
• analyze temporal patterns across multiple sequential file-print generations to identify periodic characteristics in the statistical variance between consecutive file-prints;
• and
• calculate a confidence score based on an amount of data analyzed and consistency of predictions over time;
• process the statistical file-print or incremental file-prints through a trained machine learning classifier to identify a data type;
• and
• select an encoding or decoding codebook from a plurality of codebooks based on the identified data type, wherein the selection for streaming data can be dynamically adjusted in response to detected changes in data type within the continuous stream.