Locating potentially identical objects across multiple computers based on stochastic partitioning of workload

One or more computer storage media having stored thereon a plurality of instructions that, when executed by one or more processors of a computer that is one of a plurality of computers in a network, causes the one or more processors to perform the following acts:

• selecting ones of the plurality of computers to populate a plurality of groups, wherein the selecting is based at least in part on the number of computers in the network that the computer is aware of;
• selecting a plurality of bits of file information corresponding to a file, wherein the plurality of bits of file information comprises a set of least significant bits of the file information and a number of bits in the set of least significant bits is based at least in part on the number of other computers in the network of which the computer is aware;
• and
• identifying which of the selected ones of the plurality of computers to communicate the file information to for identification of potentially identical files on the plurality of computers, wherein the identifying is based at least in part on comparing the selected plurality of bits of the file information to a corresponding plurality of bits of a computer identifier of one or more of the selected ones of the plurality of computers.

A computing device that facilitates locating potentially identical objects across a plurality of computing devices, wherein the computing device is one of the plurality of computing devices, the computing device comprising:

• a processing unit;
• a memory coupled to the processing unit;
• a distributed file system interface connecting the computing device to a network, wherein the network comprises the plurality of computing devices;
• a grouping module stored in the memory and executed on the processing unit to determine a plurality of groups of the plurality of computing devices, wherein criteria for determining the plurality of groups comprises:
• determining a number of computing devices of which the computing device is aware;
• and
• determining a number of the plurality of groups of the plurality of computing devices as a function of the number of computing devices of which the computing device is aware;
• a file information generation module stored in the memory and executed on the processing unit to generate object information for use in locating potentially identical objects across the plurality of computing devices;
• a forwarding location determination module stored in the memory and executed on the processing unit to
• perform a bit level comparison of the object information and a computer identifier representing a particular one of the plurality of computing devices;
• and
• in an event that the object information matches the computer identifier:
• identifying a particular one of the plurality of groups to which the particular one of the plurality of computing devices belongs;
• facilitating each of the plurality of computing devices that belongs to the particular one of the plurality of groups to determine potentially identical objects.