System, method, and computer program product for crawling a website based on a scheme of the website
AssigneeMcAfee, Inc.
InventorGabriel Pack
Life of this patent
Estimating the term…
Abstract
A system, method, and computer program product are provided for crawling a website based on a scheme of the website. In use, a difference between a first content and second content of a website is identified. Additionally, a scheme of the website is identified based on the difference. Furthermore, the website is crawled based on the scheme.
Claims20
A non-transitory computer readable medium comprising instructions stored thereon that when executed by one or more processors cause the one or more processors to:
- determine a difference between a first content and a second content of a single website, the first content and second content identified in a first and second probe of the single website, each probe comprising a request and a corresponding response for information;
- identify a scheme of the single website based on the difference;
- and
- direct crawling of a portion of the single website, the directed crawling based on the identified scheme, and the directed crawling to further identify vulnerability to unwanted activity on the single website.
A method, comprising:
- determining, utilizing a hardware processor, a difference between a first content and a second content of a single website, the difference determined in part by first removing at least one common sequence from said first content and said second content, the first content and second content identified in a first and second probe of the single website, each probe comprising a request and a corresponding response for information;
- identifying a scheme of the single website based on the difference;
- and
- directing crawling of a portion of the single website, the crawling directed based on the scheme, and the directed crawling to identify vulnerability to unwanted activity on the single website.
A non-transitory computer readable medium comprising instructions stored thereon that when executed cause one or more processors to:
- remove at least one common sequence from a first content and a second content of a single website, the first content and second content retrieved in response to a first and second probe of the single website, each probe comprising a request and a corresponding response for information;
- determine a difference between said first and second content after said at least one common sequence has been removed from said first and second content;
- identify a scheme of the single website based on the difference;
- and
- direct crawling of a portion of the single website, the directed crawling based on the scheme, and the directed crawling to identify vulnerability to unwanted activity on the single website.
Dependencies are inferred from the claim text.