# US 2014/0156647 A1 · Phrase-based detection of duplicate documents in an information retrieval system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2014/0156647 A1
- Document ID: US20140156647A1
- Status: Published application
- Kind: A1
- Application number: 13/919,830
- Filed: 2013-06-17
- Published: 2014-06-05
- Earliest claimed: 2004-07-26
- Assignees: Google Inc.
- Applicants: Google Inc.
- Inventors: Anna L. Patterson
- Web page: https://patentgrep.com/patent/US20140156647A1

## CPC

- G06F 16/3344: {using natural language analysis}
- G06F 16/2237: {Vectors, bitmaps or matrices}
- G06F 16/243: {Natural language query formulation}
- G06F 16/24578: {using ranking}
- G06F 16/3322: {using system suggestions (G06F16/3325 takes precedence)}
- G06F 16/93: Document management systems
- G06F 16/951: Indexing; Web crawling techniques
- G06Q 10/10: Office automation; Time management
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

An information retrieval system uses phrases to index, retrieve, organize and describe documents. Phrases are identified that predict the presence of other phrases in documents. Documents are the indexed according to their included phrases. Related phrases and phrase extensions are also identified. Phrases in a query are identified and used to retrieve and rank documents. Phrases are also used to cluster documents in the search results, create document descriptions, and eliminate duplicate documents from the search results, and from the index.

## Claims

### Claim 1

A method of detecting duplicate documents in search results, the method comprising:
receiving a query comprising at least one phrase;
retrieving a plurality of documents responsive to the query to form a search result;
for each of the retrieved documents, generating a document description comprising selected sentences of the document, wherein the selected sentences are ordered in the document description as a function of a number of phrases in each sentence;
responsive to the document description at least two documents matching, discarding at least one of the two documents from the search result.

### Claim 2

A method of detecting duplicate documents in search results, the method comprising:
receiving a query comprising at least one phrase;
retrieving a plurality of documents responsive to the query to form a search result;
for each of the retrieved documents, retrieving a stored document description comprising selected sentences of the document, wherein the selected sentences are ordered in the document description as a function of a number of phrases in each sentence;
responsive to the document description at least two documents matching, discarding at least one of the two documents from the search result.