US 2016/0103831 A1 · DETECTING HOMOLOGIES IN ENCRYPTED AND UNENCRYPTED DOCUMENTS USING FUZZY HASHING

Abstract

Techniques are provided for automatically detecting homologies between documents based on structural characteristics. Various statistics relating to the COS structure of a PDF document are compiled. The statistics are input into a rolling hash function to generate a digital fingerprint of the document. Fingerprints from two similar documents will have small edit distances between them, and can therefore be classified similarly or provided as results to a fingerprint-based search. For example, an unclassified document may be classified in the same class as a representative document where the fingerprints of the two documents have a small edit distance between them. Since the structure of the document is used instead of the text content or renderings, it is possible to operate on encrypted documents. Further, representative elements of a particular class of documents can be selected for comparison against a target document for improved resolution of the results.

Claims20

Dependent claims

A computer-implemented method comprising:

compiling, by a processor, a first set of statistics relating to a first structure of a first electronic document;
computing, by the processor, a first hash on the first set of statistics;
compiling, by the processor, a second set of statistics relating to a second structure of a second electronic document;
computing, by the processor, a second hash on the second set of statistics ;
and
determining, by the processor, an edit distance between the second hash and the first hash.

3 more elements

A system comprising:

a storage;
a processor operatively coupled to the storage, the processor configured to execute instructions stored in the storage that when executed cause the processor to carry out a process comprising:
compiling a first set of statistics relating to a first structure of a first electronic document;
computing a first hash on the first set of statistics using a rolling hash function;
compiling a second set of statistics relating to a second structure of a second electronic document;
computing a second hash on the second set of statistics using the rolling hash function;
and
determining an edit distance between the second hash and the first hash.

5 more elements

A non-transient computer program product having instructions encoded thereon that when executed by one or more processors cause a process to be carried out, the process comprising:

compiling a first set of statistics relating to a first structure of a first electronic document;
computing a first hash on the first set of statistics using a rolling hash function;
compiling a second set of statistics relating to a second structure of a second electronic document;
computing a second hash on the second set of statistics using the rolling hash function;
and
determining an edit distance between the second hash and the first hash.

3 more elements

Dependencies are inferred from the claim text.

Detecting homologies in encrypted and unencrypted documents using fuzzy hashing

Life of this patent

Detecting homologies in encrypted and unencrypted documents using fuzzy hashing

Life of this patent

Abstract

Claims20