Matching funnel for large document index

A computerized system including at least one processor and one or more computer storage media, the system including:

• a query reformulation component that analyzes a received search query to identify one or more atoms based on terms contained in the received search query and generates a reformulated query;
• a document matching component that queries a search index using the reformulated query to identify an initial set of matching documents, the search index storing a plurality of atoms, the plurality of atoms comprising one or more n-grams, one or more n-tuples, and one or more near n-tuples;
• a document pruning component that computes a preliminary score for each document from the initial set of matching documents using both a simplified scoring function and pre-computed scores stored in the search index for document/atom pairs for the one or more atoms and the initial set of documents, and identifies a pruned set of documents based on the preliminary scores;
• and
• a final document ranking component that computes a ranking score for each document in the pruned set of documents using a full ranking algorithm.

A method for providing search results in response to a search query using a staged process comprising:

• receiving a search query;
• identifying one or more atoms from the search query;
• identifying an initial set of documents containing the one or more atoms from a search index, the search index storing a plurality of atoms, the plurality of atoms comprising one or more n-grams, one or more n-tuples, and one or more near n-tuples;
• computing a preliminary score for each document in the initial set of documents using both a simplified scoring function and pre-computed scores stored in the search index for document/atom pairs for the one or more atoms and the initial set of documents;
• selecting a subset of documents for further processing based on the preliminary scores;
• computing a ranking score for each document in the subset of documents using a final ranking algorithm;
• and
• providing a set of search results based on the ranking scores.