Systems and methods for facilitating the gathering of open source intelligence

A system for use in extracting content of interest from a collection of webpages, the system comprising:

• a processing module;
• and
• a memory module logically connected to the processing module and comprising a set of computer readable instructions executable by the processing module to:
• acquire source code used to generate each webpage of a collection of webpages on a display;
• obtain a similarity matrix for the collection of webpages from the acquired source code using a text-based similarity measure;
• use the similarity matrix to determine a maximum dissimilarity score for each webpage in relation to the collection of webpages;
• and
• present, on a display, one or more lists of those webpages associated with maximum dissimilarity scores above a threshold dissimilarity score.

A system for use in extracting a textual hierarchy from one or more webpages of a website for use in creating a hierarchical signature for the website, the system comprising:

• a processing module;
• and
• a memory module logically connected to the processing module and comprising a set of computer readable instructions executable by the processing module to:
• receive the x most frequently disclosed terms among one or more pages of a website, wherein x is a positive integer;
• first cluster the x most frequently disclosed terms into two or more sets of terms as a function of semantic similarity between the terms in each subset;
• second cluster the two or more sets of terms into two or more subsets of terms utilizing contextual information located proximate each of the terms, wherein each of the terms in the subsets comprises a lower level term;
• and
• ascertain, for each of the two or more subsets, an upper level term that semantically encompasses each of the lower level terms in the subset.