False negative prediction for training a machine-learning model

A method, performed by a computing system comprising a processor and a non-transitory computer-readable medium, comprising:

• initializing a set of parameters for a machine-learning model;
• accessing search log data captured by an online system, wherein the search log data describes a plurality of queries placed by users of the online system and a plurality of items;
• generating a set of initial training examples based on the search log data, wherein each initial training example comprises a query of the plurality of queries, an item of the plurality of items, and an initial label, wherein the initial label represents whether the item was presented as a search result to a user and was selected by the user;
• identifying a set of initial negative examples by identifying a subset of the set of initial training examples with initial labels below a threshold value;
• updating the set of initial negative examples to generate a set of updated negative examples, wherein the set of updated negative examples is generated by, for each initial negative example:
• identifying a plurality of bridge queries for the query of the initial negative example, wherein the plurality of bridge queries is a subset of the plurality of queries for which the item of the initial negative example was presented as a search result to a user and was selected by the user;
• computing a similarity score between the query of the initial negative example and each of the plurality of bridge queries;
• computing an updated label for the initial negative example based on the computed similarity scores;
• and
• generating an updated negative example comprising the computed updated label and the query and item of the initial negative example;
• generating a final set of training examples comprising the set of updated negative examples and a subset of the set of initial training examples with initial labels above the threshold value;
• updating the set of parameters by processing each training example in the final set of training examples, wherein updating the set of parameters results in an updated set of parameters for the machine-learning model, and wherein processing each training example in the set of training examples comprises:
• applying the machine-learning model to the item data and the query of the training example to generate an item prediction score, wherein the item prediction score represents a predicted likelihood that a user would select the item when the item is presented as a search result for the query;
• computing a loss score by comparing the item prediction score to a label of the training example;
• and
• updating the set of parameters for the machine-learning model through a backpropagation process using the computed loss score;
• and
• storing the updated set of parameters on the computer-readable medium.

A system comprising:

• a processor;
• and
• a non-transitory computer-readable medium storing instructions that, when executed by the processor, cause the processor to perform operations comprising:
• initializing a set of parameters for a machine-learning model;
• accessing search log data captured by an online system, wherein the search log data describes a plurality of queries placed by users of the online system and a plurality of items;
• generating a set of initial training examples based on the search log data, wherein each initial training example comprises a query of the plurality of queries, an item of the plurality of items, and an initial label, wherein the initial label represents whether the item was presented as a search result to a user and was selected by the user;
• identifying a set of initial negative examples by identifying a subset of the set of initial training examples with initial labels below a threshold value;
• updating the set of initial negative examples to generate a set of updated negative examples, wherein the set of updated negative examples is generated by, for each initial negative example:
• identifying a plurality of bridge queries for the query of the initial negative example, wherein the plurality of bridge queries is a subset of the plurality of queries for which the item of the initial negative example was presented as a search result to a user and was selected by the user;
• computing a similarity score between the query of the initial negative example and each of the plurality of bridge queries;
• computing an updated label for the initial negative example based on the computed similarity scores;
• and
• generating an updated negative example comprising the computed updated label and the query and item of the initial negative example;
• generating a final set of training examples comprising the set of updated negative examples and a subset of the set of initial training examples with initial labels above the threshold value;
• updating the set of parameters by processing each training example in the final set of training examples, wherein updating the set of parameters results in an updated set of parameters for the machine-learning model, and wherein processing each training example in the set of training examples comprises:
• applying the machine-learning model to the item data and the query of the training example to generate an item prediction score, wherein the item prediction score represents a predicted likelihood that a user would select the item when the item is presented as a search result for the query;
• computing a loss score by comparing the item prediction score to a label of the training example;
• and
• updating the set of parameters for the machine-learning model through a backpropagation process using the computed loss score;
• and
• storing the updated set of parameters on the computer-readable medium.