Large language model engine analysis and tracking of dark web data and threat actors

A system comprising:

• at least one computer processor;
• a data repository, in communication with the at least one computer processor and stored on a physical storage device, configured to store a plurality of raw dark web posts;
• and
• a dark web analyzer executing on the at least one computer processor, wherein:
• the dark web analyzer is configured to cause the at least one computer processor to
• obtain the plurality of raw dark web posts, each raw dark web post of the plurality of raw dark web posts including a natural language utterance,
• cause a preprocessor executing on the at least one computer processor to preprocess the plurality of raw dark web posts to obtain a tokenized dark web post set,
• construct a prompt for a threat analyzer LLM, including at least a first tokenized dark web post of the tokenized dark web post set;
• a set of threat personas, each threat persona of the set of threat personas comprising a category and a definition, and
• an instruction to categorize the first tokenized dark web post with a threat persona of the set of threat personas and generate a first explanation corresponding to the categorization of the first tokenized dark web post;
• cause the threat analyzer LLM executing on the at least one computer processor to process the prompt, to obtain a first annotated post and a corresponding first threat persona category,
• wherein the first annotated post includes a post summary corresponding to the first tokenized dark web post, and
• the first explanation, wherein the first explanation is based on a first threat persona definition;
• and
• add the first annotated post and the corresponding first threat persona category to a post-persona catalog in the data repository.

A method, comprising:

• obtaining, by a dark web analyzer, a plurality of raw dark web posts from a data repository on a physical storage device, each raw dark web post of the plurality of raw dark web posts including a natural language utterance;
• performing, by a preprocessor, a set of preprocessing operations on each raw dark web post of the plurality of raw dark web posts to obtain a corresponding plurality of preprocessed dark web posts, wherein the set of preprocessing operations includes translation, stop word removal, and noise removal;
• classifying a subset of the plurality of preprocessed dark web posts, wherein each preprocessed dark web post of the subset of preprocessed dark web posts is classified into one of a relevant class and a non-relevant class, to obtain a post classifier training dataset;
• training, by the dark web analyzer, a post classifier ML model with the post classifier training dataset, to obtain a trained post classifier ML model, wherein the post classifier ML model is a Gaussian Naïve Bayes model;
• and
• classifying, by the trained post classifier ML model, each preprocessed dark web post of the plurality of preprocessed dark web posts, into one of the relevant class and the non-relevant class, to obtain a corresponding plurality of classified dark web posts.