Method and system for constructing a document redundancy graph

A system for constructing a document redundancy graph, said system comprising:

• a processor;
• a data bus coupled to said processor;
• and
• a computer-usable mass storage device embodying computer code, said computer-usable medium being coupled to said data bus, said computer program code comprising instructions executable by said processor and configured for:
• representing each paragraph associated with a document set as a node among a plurality of nodes, wherein each node among said plurality of nodes with respect to said redundancy graph represents a unique cluster of information related to said each paragraph;
• providing said each paragraph with a unique paragraph identifier;
• constructing a hash table of all paragraph identifiers comprising identifiers of all paragraphs reachable from said each paragraph;
• merging said plurality of nodes associated with redundant information by configuring said hash table with respect to a pair of paragraph identifiers in association with a probability value, wherein said probability value sorts a plurality of information matches in an order of decreasing certainty of common content, wherein a pair of said paragraph identifiers associated with an increased certainty of common content are selected to merge;
• and
• combining said plurality of nodes unique to a single document by expressing a pair of nodes with overlapping common content as a combined node, wherein said combined node comprises an empty intersection of said pair of nodes and comparing each paragraph identifier among said pair of paragraph identifiers to a probability value associated with an entry in said hash table in an order wherein said hash table eliminates inconsistency associated with said plurality of information matches.

A computer-usable mass storage for constructing a document redundancy graph, said computer-usable mass storage storing computer program code, said computer program code comprising program instructions executable by a processor, said program instructions comprising:

• program instructions to represent each paragraph associated with a document set as a node among a plurality of nodes, wherein each node among said plurality of nodes with respect to said redundancy graph represents a unique cluster of information related to said each paragraph;
• program instructions to provide said each paragraph with a unique paragraph identifier;
• program instructions to construct a hash table of all paragraph identifiers comprising identifiers of all paragraphs reachable from said each paragraph;
• program instructions to merge said plurality of nodes associated with redundant information by configuring said hash table with respect to a pair of paragraph identifiers in association with a probability value, wherein said probability value sorts a plurality of information matches in an order of decreasing certainty of common content, wherein a pair of said paragraph identifiers associated with an increased certainty of common content are selected to merge;
• and
• program instructions to combine said plurality of nodes unique to a single document by expressing a pair of nodes with overlapping common content as a combined node, wherein said combined node comprises an empty intersection of said pair of nodes and comparing each paragraph identifier among said pair of paragraph identifiers to a probability value associated with an entry in said hash table in an order wherein said hash table eliminates inconsistency associated with said plurality of information matches.