US 8,527,534 B2Granted

Bootstrap and adapt a document search engine

AssigneeMicrosoft Corporation

InventorsKuansan Wang, Bo-June Hsu, Xiaolong Li

Life of this patent

Estimating the term…

Loading patent…

Abstract

Architecture that employs a modeling technique based on language modeling to estimate a probability of a document matching the user need as expressed in the query. The modeling technique is based on the data mining results that various portions of a document (e.g., body, title, URL, anchor text, user queries) use different styles of human languages. Thus, the results based on a language can be adapted individually to match the language of query. Since the approach is based on adaptation, the framework also provides a natural means to progressively revise the model as user data are collected. Different styles of languages in a document can be recognized and adapted individually. Background language models are also employed that offer a fallback approach in case the document has incomplete fields of data, and can utilize topical or semantic hierarchy of the knowledge domain.

Claims20

Dependent claims

A computer-implemented information retrieval system comprising:

a query component configured to receive and process a query to retrieve a document having different language styles as data streams, the data streams including at least a first stream for a first portion of the document and a second stream for a second portion of the document, wherein the second stream is different from the first stream and the second portion is different from the first portion;
a model component configured to create multiple language models for the data streams of the document, the multiple language models characterizing the data streams for document ranking, the multiple language models comprising at least a first language model for the first portion of the document and a second language model for the second portion of the document, wherein the second language model is different from the first language model;
an estimation component configured to estimate mixture weights for the multiple language models;
a smoothing component configured to apply smoothing to the multiple language models to generate similar cardinality among the multiple language models for linearly combining scores output by the multiple language models;
and
a processor configured to execute the query component, the model component, the estimation component, and the smoothing component.

3 more elements

One or more hardware computer-readable memory devices or hardware storage devices comprising executable instructions which, when executed by one or more processors, cause the one or more processors to perform acts comprising:

identifying data streams in different portions of a document associated with a query, the data streams including at least a first data stream obtained from a first portion of the document and a second data stream that is different from the first data stream, the second data stream being obtained from a second portion of the document other than the first portion;
modeling the data streams as separate language models corresponding to the different portions of the document, the separate language models including at least:
a first language model of the first data stream obtained from the first portion of the document, the first language model reflecting first language usage in the first portion of the document, and
a second language model, different from the first language model, of the second data stream obtained from the second portion of the document, the second language model reflecting second language usage in the second portion of the document that is different from the first language usage in the first portion of the document;
estimating mixture weights for combining the separate language models into a mixed language model;
and
creating the mixed language model by interpolating the mixture weights.

4 more elements

A system comprising:

one or more processors;
and
computer-executable instructions that, when executed by the one or more processors, configure the one or more processors to:
identify data streams in different portions of a document, the data streams including at least a first data stream obtained from a first portion of the document and a second data stream that is different from the first data stream, the second data stream being obtained from a second portion of the document other than the first portion;
model the data streams as separate language models corresponding to the different portions of the document, the separate language models including at least:
a first language model of the first data stream obtained from the first portion of the document, the first language model reflecting first language usage in the first portion of the document, and
a second language model of the second data stream obtained from the second portion of the document, wherein the second language model is different than the first language model and reflects second language usage in the second portion of the document that is different from the first language usage in the first portion of the document;
and
use the separate language models to estimate a probability that the document matches a query by:
adapting the separate language models individually to match the query, and
characterizing the document as a mixture model that combines the separate language models.

8 more elements

Dependencies are inferred from the claim text.