Information retrieval systems with duplicate document detection and presentation functions

An information-retrieval system comprising:

• a plurality of databases;
• and
• a server for providing users access to one or more of the databases, the server including:
• query-definition means for defining a query and selecting an option related to identification of documents within results of the query that include content duplicative of content within one or more other documents within the results;
• duplicate-determination means for determining whether one or more documents within the results of the query include content duplicative of content within one or more other documents within the results;
• means for controlling display of results of the query based on the selected option, with at least one of the displayed results indicated as including content duplicative of content in one or more other documents within the results;
• and
• means for controlling output of results of the query to a printer or email transmission device, based on user selected options related to output of documents that include content duplicative of content of one or more other documents within the results.

A method comprising:

• comparing first and second lengths of respective first and second documents;
• comparing first and second content sets for the respective first and second documents;
• and
• determining whether the first and second documents are duplicates based on results of comparing the first and second lengths or results of comparing the first and second content sets.

A method of identifying whether first and second documents are likely to contain duplicate content, the method comprising:

• determining whether the first and second documents have corresponding temporal traits that are within a first range of each other;
• determining whether the first and second documents have corresponding length traits that are within a second range of each other;
• determining whether the first and second documents have a significant number of features in common with each other;
• and
• identifying the first and second documents as duplicates in response to determining that the temporal traits of the first and second documents are within the first range of each other, that the lengths of the first and second documents are within the second range of each other, and that the first and second documents have at least a significant number of features in common with each other.

A method comprising:

• receiving a user query;
• identifying at least first and second documents from a database in response to the user query, with the first and second documents associated with respective first and second feature vectors, each feature vector having a plurality of equal-length binary representations of terms or features within its respective document, with the terms or features selected based on relative magnitude of corresponding inverse-document-frequency (idf) values within a table of inverse-document-frequency values;
• and
• in response to identifying the first and second documents, comparing the first and second feature vectors.