Method and apparatus for finding mirrored hosts by analyzing urls
AssigneeAlta Vista Company
InventorsKrishna A. Bharat, Andrei Broder, Steven C. Glassman, Jeffrey Dean, Monika R. Henzinger
Life of this patent
Estimating the term…
Abstract
A method and apparatus that detects mirrored host pairs using information about a large set of pages, including URLs. The identities of the detected mirrored hosts are then saved so that browsers, crawlers, proxy servers, or the like can correctly identify mirrored web sites. The described embodiments of the present invention look at the URLs of pages hosts to determine whether the hosts are potentially mirrored.
Claims21
A method of determining mirrored web hosts, comprising:
- receiving information about the addresses of a plurality of web sites stored on a plurality of hosts;
- determining a plurality of terms of the URLs associated with every host;
- weighting the terms in inverse proportion to frequency;
- determining a similarity score for host pair in accordance with the weighted terms;
- and
- outputting a list of potential pairs of mirrored hosts in accordance with their similarity scores.
An apparatus that determines mirrored web hosts, comprising:
- software configured to receive information about the addresses of a plurality of web sites stored on a plurality of hosts;
- software configured to determine a plurality of terms of the URLs associated with every host;
- software configured to weight the terms in inverse proportion to frequency;
- software configured to determine a similarity score for host pair in accordance with the weighted terms;
- and
- software configured to output a list of potential pairs of mirrored hosts in accordance with their similarity scores.
A computer program product, comprising:
- computer program code devices configured to receive information about the addresses of a plurality of web sites stored on a plurality of hosts;
- computer program code devices configured to determine a plurality of terms of the URLs associated with every host;
- computer program code devices configured to weight the terms in inverse proportion to frequency;
- computer program code devices configured to determine a similarity score for host pair in accordance with the weighted terms;
- and
- computer program code devices configured to output a list of potential pairs of mirrored hosts in accordance with their similarity scores.
Dependencies are inferred from the claim text.