Ranking Test Framework for Search Results on an Online Social Network

A system comprising:

• one or more processors of a computing system;
• and a memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
• access, by the computing system, a first set of scored results received from a client system of a first user, the first set of scored results comprising:
• one or more results generated by a first search algorithm of a search-engine system in response to a query from the first user, each result having a rank with respect to the other results;
• and
• one or more scores inputted by the first user corresponding to the one or more results, respectively;
• calculate, by the computing system, a discounted cumulative gain for each result in the first set of scored results based on the rank of the result and the score inputted by the first user corresponding to the result;
• generate, by the computing system, a second search algorithm based at least in part on the calculated discounted cumulative gains and the first search algorithm, wherein the second search algorithm comprises a modification of the first search algorithm to improve a ranking of the results;
• and
• if a quality metric of the second search algorithm is higher than a quality metric of the first search algorithm, then substitute the second search algorithm at the search-engine system for the first search algorithm;
• else, retain the first search algorithm at the search-engine system.

One or more computer-readable non-transitory storage media embodying software that is operable when executed by a computing system to:

• access, by the computing system, a first set of scored results received from a client system of a first user, the first set of scored results comprising:
• one or more results generated by a first search algorithm of a search-engine system in response to a query from the first user, each result having a rank with respect to the other results;
• and
• one or more scores inputted by the first user corresponding to the one or more results, respectively;
• calculate, by the computing system, a discounted cumulative gain for each result in the first set of scored results based on the rank of the result and the score inputted by the first user corresponding to the result;
• generate, by the computing system, a second search algorithm based at least in part on the calculated discounted cumulative gains and the first search algorithm, wherein the second search algorithm comprises a modification of the first search algorithm to improve a ranking of the results;
• and
• if a quality metric of the second search algorithm is higher than a quality metric of the first search algorithm, then substitute the second search algorithm at the search-engine system for the first search algorithm;
• else, retain the first search algorithm at the search-engine system.