US 8,713,071 B1Granted

Detecting mirrors on the web

AssigneeGoogle Inc.

InventorArvind Jain

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system obtains multiple signals associated with first and second hostnames or subtrees. The system uses the multiple signals to determine whether the first hostname or subtree is a mirror of the second hostname or subtree. The multiple signals may include, for example, a same link structure and/or sitemap for the first and second hostnames or subtrees, duplicate content associated with the first and second hostnames or subtrees, a same Internet Protocol (IP) address or subnet for the first and second hostnames or subtrees, a same owner for the first and second hostnames or subtrees, nearly duplicate content associated with the first and second hostnames or subtrees, similarity between the hostnames of the first and second hostnames or subtrees; and/or an indication that one hostname or subtree of the first and second hostnames or subtrees redirects to the other hostname or subtree.

Claims23

Dependent claims

A computer-implemented method comprising:

identifying, by a processor associated with the computer, path components of addresses that are associated with a group of hosts;
reversing, by a processor associated with the computer, the path components;
sorting, by a processor associated with the computer, the addresses based on the reversed path components;
identifying, by a processor associated with the computer, an adjacent pair of addresses from the sorted addresses,
the adjacent pair of addresses including:
a first address associated with a first host of the group of hosts, and
a second address associated with a second host of the group of hosts;
identifying, by a processor associated with the computer and based on identifying the adjacent pair of addresses, the first host as a potential mirror of the second host that differs from the first host;
determining, by a processor associated with the computer, first information associated with the first host and second information associated with the second host;
calculating, by a processor associated with the computer, a score based on the first information and the second information;
determining, by a processor associated with the computer, whether to classify the first host as an actual mirror of the second host,
the first host being classified as the actual mirror of the second host when the score satisfies a particular threshold value;
and
storing, by a processor associated with the computer, information identifying whether the first host is the actual mirror of the second host.

11 more elements

A system comprising:

one or more devices, at least partially implemented in hardware, configured to:
identify path components of addresses that are associated with a group of hosts;
reverse the path components,
sort the addresses based on the reversed path components;
identify an adjacent pair of addresses from the sorted addresses,
the adjacent pair of addresses including:
a first address associated with a first host of the group of hosts, and
a second address associated with a second host of the group of hosts;
identify, based on identifying the adjacent pair of addresses, the first host as a potential mirror of the second host that differs from the first host;
determine first information that is associated with the first host and second information that is associated with the second host;
calculate a score based on the first information and the second information;
and
determine whether the first host is an actual mirror of the second host based on the score,
the first host being the actual mirror of the second host when the score satisfies a particular threshold value.

11 more elements

A non-transitory memory device to store instructions, the instructions comprising:

one or more instructions which, when executed by one or more processors, cause the one or more processors to identify path components of addresses that are associated with a group of hosts;
one or more instructions which, when executed by the one or more processors, cause the one or more processors to reverse the path components;
one or more instructions which, when executed by the one or more processors, cause the one or more processors to sort the addresses based on the reversed path components;
one or more instructions which, when executed by the one or more processors, cause the one or more processors to identify an adjacent pair of addresses from the sorted addresses,
the adjacent pair of addresses including:
a first address associated with a first host of the group of hosts, and
a second address associated with a second host of the group of hosts;
one or more instructions which, when executed by the one or more processors, cause the one or more processors to identify the first host as a potential mirror of the second host that differs from the first host based on identifying the adjacent pair of addresses;
one or more instructions which, when executed by the one or more processors, cause the one or more processors to determine first information associated with the first host and second information associated with the second host;
one or more instructions which, when executed by the one or more processors, cause the one or more processors to calculate a score based on the first information and the second information;
and
one or more instructions which, when executed by the one or more processors, cause the one or more processors to determine whether to classify the first host as an actual mirror of the second host,
the first host being classified as the actual mirror of the second host when the score satisfies a particular threshold value.

10 more elements

Dependencies are inferred from the claim text.

Facts

Application: 13/289,688Filed 4 Nov 2011 · continuation
Priority: 9 Aug 2005
Granted: 29 Apr 2014
Term ends: Not estimatedThe source records don’t support an estimate.

Classification4 codes

H04L 67/1095
In which an application is distributed across nodes in the network: replication or mirroring of data, e.g. scheduling or transport for data…
G06F 16/95
Details of database functions independent of the retrieved data types: retrieval from the web
H04L
Digital communication
G06F
Computing and data processing