Learning Element Weighting for Similarity Measures
AssigneeMicrosoft Technology Licensing, LLC
InventorsWen-tau Yih, Christopher A. Meek, Hannaneh Hajishirzi
Life of this patent
Estimating the term…
Abstract
Described is a technology for measuring the similarity between two objects (e.g., documents), via a framework that learns the term-weighting function from training data, e.g., labeled pairs of objects, to develop a learned model. A learning procedure tunes the model parameters by minimizing a defined loss function of the similarity score. Also described is using the learning procedure and learned model to detect near duplicate documents.
Claims20
In a computing environment, a method performed on at least one processor, comprising:
- creating a similarity measure between objects, each object represented by vectors, including by learning values of vector elements based upon:
- a functional form of the element value with parameters to learn, a similarity function for computing a similarity score given the vectors, and a loss function of computed similarity scores and labels of the vectors;
- and
- tuning the parameters in the similarity function to decide the element values, including by minimizing the loss function.
In a computing environment, a system comprising, a model that produces a similarity measure between a set of objects, the model learned via a framework that represents objects as vectors and learns values of vector elements by accessing labeled training data and using the labeled training data to learn model parameters that minimize a loss function.
One or more computer-readable media having computer-executable instructions, which when executed perform steps, comprising:
- extracting terms from an object,
- generating features from the terms, and including the features in term vectors representing that object;
- applying a learned term-weighting function to determine a term-weighting score of each term;
- and
- computing a similarity score based upon the term vector.
Dependencies are inferred from the claim text.