Answer model comparison
AssigneeMicrosoft Corporation
InventorsTapas Kanungo, Kuansan Wang, Ran Gilad-Bachrach, Kieran McDonald, Kumaresh Pattabiraman, Christopher Meyers, Ashok Ponnuswami, Luke Simon
Life of this patent
Estimating the term…
Abstract
This patent application pertains to answer model comparison. One implementation can determine a first frequency at which an individual answer category appears in an individual slot on a query results page when utilizing a first model. The method can ascertain a second frequency at which the individual answer category appears in the individual slot on the query results page when utilizing a second model. The method can calibrate the second model so that the second frequency approaches the first frequency.
Claims20
A method implemented by at least one computing device, the method comprising:
- determining a first frequency at which an individual answer category appears in an individual first slot on a first query results page that is generated utilizing a first model;
- ascertaining a second frequency at which the individual answer category appears in an individual second slot on a second query results page that is generated utilizing a second model, the individual second slot on the second query results page corresponding to the individual first slot on the first query results page;
- and,
- calibrating the second model so that the second frequency at which the individual answer category appears in the individual second slot on the second query results page approaches the first frequency at which the individual answer category appears in the individual first slot on the first query results page.
A system, comprising:
- at least one processor;
- and,
- an answer model calibration module that is executable by the at least one processor and that is configured to maintain one or more characteristics of a first search results page that is produced utilizing a first model in a second search results page that is produced utilizing a second model,
- wherein, to maintain the one or more characteristics of the first search results page in the second search results page, the answer model calibration module is configured to calibrate one or more second thresholds applied by the second model to produce the second search results page.
At least one hardware computer-readable storage media having instructions stored thereon that when executed by a computing device cause the computing device to perform acts, the acts comprising:
- sampling queries from a query stream to form a representative query set for answer categories to be normalized;
- running an existing model that is used to generate a first query results page through the representative query set to find first placement scores of individual answer categories at individual first slots of the first query results page, the individual first slots having associated first thresholds in the existing model;
- and,
- calibrating a new model that is used to generate a second query results page using the first placement scores.
Dependencies are inferred from the claim text.