Flexible deterministic finite automata (dfa) tokenizer for ai-based malicious traffic detection

A non-transitory machine-readable medium having instructions comprising software modules stored thereon configured to be executed by one or more processors in one or more computer system, the software modules including:

• a DFA compiler, configured to process a profile comprising a plurality of strings and associated rules and generate a DFA transition table;
• a DFA engine, configured to receive an input string and generate a token sequence using the DFA transition table;
• and
• a feature extraction engine, configured to extract features from the token sequences generated by the DFA engine and convert the extracted features into feature vectors.

A Web Application Fire (WAF) comprising:

• one or more processors;
• memory, operatively coupled to the one or more processors;
• at least one network interface having one or more ports, operatively coupled to the processor;
• and
• software instructions comprising a Deterministic Finite Automata (DFA) tokenizer, a feature extraction engine, and a machine learning model configured to perform binary classification,
• where the WAF is enabled, via execution of the software instructions to:
• extract strings from inputs received at least one of the one or more ports of the network interface;
• convert the strings to token sequences using the (DFA) tokenizer;
• process the token sequences with the feature extraction engine to generate feature vectors;
• and
• classify the feature vectors using the machine learning model to determine whether inputs are malicious or benign.