US 9,519,712 B2Granted

Systems and methods to evaluate search qualities

AssigneeAT&T Intellectual Property I, L.P.

InventorsColin R. Goodall, Sylvia Halasz, Wen-Ling Hsu, Guy Joseph Jacobson, Ritu Kumar, Bradley Keith Mohs, Ann E. Skudlark

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Systems and methods are provided to evaluate a search engine using search results retrieved from a plurality of different search engines. For example, a controller can randomly distribute a set of pre-formulated queries to a plurality of computers to query the search engines at random times to obtain search results and combine the search results to evaluate the completeness of a query result provided by a particular search engine. The controller may simulate the random queries from users using a set of computers and may reduce the burden on the search engines by spreading out the query activities over a period of time, such as a month.

Claims19

Dependent claims

A method comprising:

scheduling, by a computer controller, submission of a batch of query requests to a first search engine and a plurality of second search engines, wherein the batch of query requests is submitted to the first search engine and the plurality of second search engines by a set of computers selected by the computer controller;
receiving, at the computer controller, first search results from the first search engine and second search results from the plurality of second search engines in response to submission of the batch of query requests to the first search engine and the plurality of second search engines;
aggregating, by the computer controller, the first search results received from the first search engine and the second search results received from the plurality of second search engines to form a set of aggregated unique search results, wherein aggregating the first search results received from the first search engine and the second search results received from the plurality of the second search engines to form the set of aggregated unique search results comprises eliminating an invalid result from the first search result received from the first search engine;
and
comparing, via the computer controller, the first search results received from the first search engine to the set of aggregated unique search results to determine a performance of the first search engine, wherein the performance of the first search engine is based on a completeness of the first search results received from the first search engine in comparison with the set of aggregated unique search results.

2 more elements

A computer-readable memory device storing instructions, the instructions, when executed by a computer controller, cause the computer controller to perform operations comprising:

scheduling submission of a batch of query requests to a first search engine and a plurality of second search engines, wherein the batch of query requests is submitted to the first search engine and the plurality of second search engines by a set of computers selected by the computer controller;
receiving first search results from the first search engine and second search results from the plurality of second search engines in response to submission of the batch of query requests to the first search engine and the plurality of second search engines;
aggregating the first search results received from the search engine and the second search results received from the plurality of the second search engines to form a set of aggregated unique search results, wherein aggregating the first search result received from the first search engine and the second search result received from the plurality of the second search engines to form the set if aggregated unique search results comprises eliminating an invalid result from the first search results received from the first search engine;
and
comparing the first search results received from the first search engine to the set of aggregated unique search results to determine a performance of the first search engine, wherein the performance of the first search engine is based on a completeness of the first search results received from the first search engine in comparison with the set of aggregated unique search results.

2 more elements

A computer controller comprising:

a processor;
and
a memory storing instructions that, when executed by the processor, cause the processor to perform operations comprising
scheduling submission of a batch of query requests to a first search engine and a plurality of second search engines, wherein the batch of query requests is submitted to the first search engine and the plurality of second search engines by a set of computers selected by the computer controller,
receiving first search results from the first search engine and second search results from the plurality of second search engines in response to submission of the batch of query requests to the first search engine and the plurality of second search engines,
aggregating the first search results received from the first search engine and the second search results received from the plurality of second search engines to form a set of aggregated unique search results, wherein aggregating the first search results received from the first search engine and the second search results received from the plurality of second search engines to form the set of aggregated unique search results comprises eliminating an invalid result from the first search results received from the first search engine, and
comparing the first search results received from the first search engine to the set of aggregated unique search results to determine a performance of the first search engine, wherein the performance of the first search engine is based on a completeness of the first search results received from the first search engine in comparison with the set of aggregated unique search results.

4 more elements

Dependencies are inferred from the claim text.