Hash suppression

A system, comprising:

• a hash generating engine to generate a set of hashes for one or more data objects;
• a vector engine to:
• compute a vector for each of the one or more data objects, wherein the vector represents the set of hashes for each of the one or more data objects, and
• compute a combined vector from the vectors for all of the one or more data objects, wherein the combined vector indicates how many individual data objects include each hash;
• and
• a hash suppression engine to:
• incrementally compute a hash suppression threshold,
• determine a group of hashes to be suppressed by using the hash suppression threshold, and
• suppress the group of selected hashes when performing an action.

A non-transitory machine-readable storage medium encoded with instructions executable by at least one processor, the machine-readable storage medium comprising instructions to:

• generate a set of hashes for a first object;
• compute a high-dimensional sparse vector for the first object, wherein the vector represents the set of hashes for each object;
• compute a combined high-dimensional sparse vector;
• compute a hash suppression threshold;
• consistently suppress the group of selected hashes that are above the hash suppression threshold when performing an action;
• generate a set of hashes for a second object and compute a high-dimensional sparse vector for the second object;
• update the combined high-dimensional sparse vector to include the high dimensional sparse vector for the second object;
• compute an updated hash suppression threshold;
• and
• determine an updated group of hashes to be consistently suppressed by using the updated hash suppression threshold.

The non-transitory machine-readable storage medium of claim 15, further comprising instructions to:

• compute a sum and a sum of squares of all elements of the combined high-dimensional sparse vector;
• compute a mean and a standard deviation of all elements of the combined high-dimensional sparse vector;
• and
• use the mean, the standard deviation, and a hash suppression factor to compute the hash suppression threshold.