Machine Learning and Automated Persistent Internet Domain Monitoring

A non-transitory computer-readable medium having stored thereon instructions that are executable by a computer system to cause the computer system to perform operations comprising:

• accessing a first plurality of web pages obtained by crawling an internet domain;
• parsing the first plurality of web pages to obtain an initial composite content signature, wherein content of each of the first plurality of web pages is assessed by a machine learning classifier relative to a plurality of particular categories, and wherein each of the first plurality of web pages is assigned a weighting used to contribute to the initial composite content signature;
• after a period of time has passed since first crawling the internet domain, accessing a second plurality of web pages obtained by re-crawling the internet domain;
• parsing the second plurality of web pages to obtain a second composite content signature;
• and
• comparing the initial composite content signature to the second composite content signature to determine if a threshold change has occurred for content of the internet domain.

A system, comprising:

• a processor;
• and
• a non-transitory computer-readable medium having stored thereon instructions that are executable by the system to cause the system to perform operations comprising:
• accessing a first plurality of web pages obtained by crawling an internet domain;
• parsing the first plurality of web pages to obtain an initial composite content signature, wherein content of each of the first plurality of web pages is assessed by a machine learning classifier relative to a plurality of particular categories, and wherein each of the first plurality of web pages is assigned a weighting used to contribute to the initial composite content signature;
• after a period of time has passed since first crawling the internet domain, accessing a second plurality of web pages obtained by re-crawling the internet domain;
• parsing the second plurality of web pages to obtain a second composite content signature;
• and
• comparing the initial composite content signature to the second composite content signature to determine if a threshold change has occurred for content of the internet domain.