Identifying and Linking Similar Passages in a Digital Text Corpus
AssigneeGoogle Inc.
InventorsWilliam N. Schilit, Okan Kolak, Adam Mathes
Life of this patent
Estimating the term…
Abstract
A corpus contains digital text from multiple documents. A passage mining engine identifies similar passages in the documents and stores data describing the similarities. The passage mining engine groups similar passages into groups based on degree of similarity or other criteria. The passage mining engine ranks the similar passages found in the text corpus based on quality or other criteria. A user interface is presented that includes hypertext links associated with the similar passages that allow a user to navigate the documents.
Claims43
A computer-implemented method of identifying similar passages in a plurality of documents stored in a corpus, comprising:
- building a shingle table describing shingles found in the corpus, the one or more documents in which the shingles appear, and locations in the documents where the shingles occur;
- identifying a sequence of multiple contiguous shingles that appears in a source document in the corpus and in at least one other document in the corpus;
- generating a similar passage in the source document based at least in part on the sequence of multiple contiguous shingles;
- and
- storing data describing the similar passage.
A computer program product having a computer-readable storage medium having computer executable program code embodied therein for identifying similar passages in a plurality of documents stored in a corpus, comprising:
- a shingling module configured to build a shingle table describing shingles found in the corpus, the one or more documents in which the shingles appear, and locations in the documents where the shingles occur;
- a sequencing module configured to identify a sequence of multiple contiguous shingles that appears in a source document in the corpus and in at least one other document in the corpus;
- a merging module configured to generate a similar passage in the source document based at least in part on the sequence of multiple contiguous shingles;
- and
- a data store configured to store data describing the similar passage.
A computer system for identifying similar passages in a plurality of documents stored in a corpus, comprising:
- a shingling module configured to build a shingle table describing shingles found in the corpus, the one or more documents in which the shingles appear, and locations in the documents where the shingles occur;
- a sequencing module configured to identify a sequence of multiple contiguous shingles that appears in a source document in the corpus and in at least one other document in the corpus;
- a merging module configured to generate a similar passage in the source document based at least in part on the sequence of multiple contiguous shingles;
- and
- a data store configured to store data describing the similar passage.
A computer program product having a computer-readable storage medium having executable program code stored therein, the program code comprising:
- a user interface module configured to:
- display a passage presentation region displaying a ranked list of similar passages found in a source document stored in a corpus, the similar passages in the list also found in one or more other documents stored in the corpus;
- and
- provide a hypertext link in association with a similar passage in the list that, upon selection, enables navigation to another document in the corpus in which the similar passage is found.
A computer-implemented method of identifying a similar passage in a plurality of documents stored in a corpus, comprising:
- identifying a group of words found in a source document in the corpus;
- identifying one or more target documents in the corpus that also contain the group of words;
- generating a similar passage in the source document based at least in part on the group of words found in the source document and in the one or more target documents;
- and
- storing data describing the similar passage.
A computer-implemented method of identifying a similar passage in a plurality of documents stored in a corpus, comprising:
- identifying a group of words found in a source document in the corpus;
- identifying one or more target documents in the corpus that also contain the group of words;
- generating a similar passage in the source document based at least in part on the group of words found in the source document and in the one or more target documents;
- creating a link structure including one or more links associating the similar passage in the source document with the group of words contained in the one or more target documents;
- and
- storing data describing the link structure.
Dependencies are inferred from the claim text.