System and method for detecting content similarity within email documents by sparse subset hashing

A non-transitory computer-readable medium storing program instructions that are computer-executable to:

• generate a token value for each of a plurality of character sequences of a first email document;
• generate a remainder value for each of the plurality of character sequences of the first email document by dividing each of the token values by a predetermined integer value;
• select a first subset of the plurality of character sequences of the first email document, wherein the first subset includes character sequences that have the same remainder values;
• select a second subset of the plurality of character sequences of the first email document, wherein the second subset includes character sequences that have the same remainder values, and wherein the remainder values of the second subset are different than the remainder values of the first subset
• generate a first hash value corresponding to the selected first subset of character sequences;
• generate a second hash value corresponding to the selected second subset of character sequences;
• and
• compare the first and second hash values with one or more hash values generated from a second email document.

A system for detecting content similarity in email documents comprising:

• one or more processors;
• and
• memory storing program instructions that are executable by the one or more processors to:
• generate a token value for each of a plurality of character sequences of a first email document;
• generate a remainder value for each of the plurality of character sequences of the first email document by dividing each of the token values by a predetermined integer value;
• select a first subset of the plurality of character sequences of the first email document, wherein the first subset includes character sequences that have the same remainder values;
• select a second subset of the plurality of character sequences of the first email document, wherein the second subset includes character sequences that have the same remainder values, and wherein the remainder values of the second subset are different than the remainder values of the first subset;
• generate a first hash value corresponding to the selected first subset of character sequences;
• generate a second hash value corresponding to the selected second subset of character sequences;
• and
• compare the first and second hash values with one or more hash values generated from a second email document.