System, method, and computer program product for crawling a website based on a scheme of the website
AssigneeMcAfee, Inc.
InventorGabriel Richard Pack
Life of this patent
Estimating the term…
Abstract
A system, method, and computer program product are provided for crawling a website based on a scheme of the website. In use, a difference between a first content and second content of a website is identified. Additionally, a scheme of the website is identified based on the difference. Furthermore, the website is crawled based on the scheme.
Claims21
A non-transitory computer readable medium comprising instructions stored thereon that when executed by one or more processors cause the one or more processors to:
- determine a difference between a first content and a second content of a single website, the first content and second content identified in a first crawl of the single website;
- identify a scheme of the single website based on the difference;
- and
- direct subsequent crawling of a portion of the single website, the directed crawling based on the identified scheme, and the subsequent crawling to further identify unwanted activity on the single website.
A method, comprising:
- determining, utilizing a hardware processor, a difference between a first content and a second content of a single website, the difference determined in part by first removing at least one common sequence from said first content and said second content, the first content and second content identified in a first crawl of the single website;
- identifying a scheme of the single website based on the difference;
- and
- directing subsequent crawling of a portion of the single website, the subsequent crawling directed based on the scheme, and the subsequent crawling to identify unwanted activity on the single website.
19-20. (canceled)
A non-transitory computer readable medium comprising instructions stored thereon that when executed cause one or more processors to:
- remove at least one common sequence from a first content and a second content of a single website, the first content and second content retrieved in response to a first crawl of the single website;
- determine a difference between said first and second content after said at least one common sequence has been removed from said first and second content;
- identify a scheme of the single website based on the difference;
- and
- computer code to direct subsequent crawling of a portion of the single website, the subsequent crawling based on the scheme, and the subsequent crawling to identify unwanted activity on the single website.
Dependencies are inferred from the claim text.