Processor for fast phrase searching

A method for finding phrases in a corpus of documents using a data processor, wherein the words in the corpus of documents include a set of stopwords, comprising:

• storing an index structure on a medium readable by the data processor, the index structure mapping entries in the index structure to documents in the corpus, the index structure including entries representing words found in the corpus of documents, and entries representing stopwords found in the corpus of documents, wherein one or more entries representing stopwords comprises tokens representing the corresponding stopwords coalesced with prefixes of respective adjacent words adjacent to the stopwords, the prefixes comprising one or more leading characters of, and fewer than all of characters of, the respective adjacent words;
• modifying an input phrase query provided to the data processor to form a modified query adapted to use the tokens coalesced with prefixes of respective adjacent words in the index;
• and
• executing the modified query using said index structure and the data processor.

A method for finding phrases in a corpus of documents using a data processor, wherein the words in the corpus of documents include a set of stopwords, comprising:

• parsing documents in the corpus of documents using the data processor to identify words found in the documents and the locations of the words in the documents, and adding entries representing words found in the corpus of documents to an index structure mapping entries in the index structure to documents in the corpus;
• storing the index structure in memory writable and readable by the data processor;
• and
• identifying stopwords in the set of stopwords found in documents in the corpus using the data processor, and adding entries in the index structure representing the stopwords, wherein one or more entries representing stopwords comprises tokens representing the corresponding stopwords coalesced with prefixes of, and fewer than all of characters of, respective adjacent words adjacent to the stopwords, the prefixes comprising one or more leading characters of the respective adjacent words.

An article of manufacture for use with a data processor for finding phrases in a corpus of documents, wherein the words in the corpus of documents include a set of stopwords, comprising:

• a machine readable data storage medium, instructions stored on the medium executable by the data processor to perform the steps of:
• parsing documents in the corpus of documents to identify words found in the documents and the locations of the words in the documents, and adding entries representing words found in the corpus of documents to an index structure mapping entries in the index structure to documents in the corpus;
• storing the index structure in memory writable and readable by the data processor;
• identifying stopwords in the set of stopwords found in documents in the corpus, and for each identified stopword, adding entries in the index structure representing the identified stopwords, wherein one or more entries representing stopwords comprises tokens representing the corresponding stopwords coalesced with prefixes of respective adjacent words adjacent to the stopwords, the prefixes comprising one or more leading characters of, and fewer than all of characters of, the respective adjacent words;
• modifying an input phrase query provided to the data processor to form a modified query with a search token representing the stopword coalesced with the prefix of an adjacent word in the subject phrase;
• and
• executing the modified query using said index structure.