US 2020/0005149 A1Published application

Applying learning-to-rank for search

AssigneeMicrosoft Technology Licensing, LLC

InventorsRohan Ramanath, Gungor Polatkan, Qi Guo, Cagri Ozcaglar, Krishnaram Kenthapadi, Sahin Cem Geyik

Life of this patent

Estimating the term…

Loading patent…

Abstract

Techniques for applying learning-to-rank with deep learning models for search are disclosed herein. In some embodiments, a computer system trains a ranking model using training data and a loss function, with the ranking model comprising a deep learning model and being configured to generate similarity scores based on a determined level of similarity between profile data of reference candidates users in the training data and reference query data of reference queries in the training data. The computer system receives a target query comprising target query data from a computing device of a target querying user, and then generates a corresponding score for target candidate users based on a determined level of similarity between profile data of the target candidate users and the target query data using the trained ranking model.

Claims20

Dependent claims

A computer-implemented method comprising:

receiving, by a computer system having a memory and at least one hardware processor, training data comprising a plurality of reference queries, a plurality of reference search results for each one of the plurality of reference queries, a plurality of user actions for each one of the plurality of reference queries, and a corresponding reaction indication for each one of the plurality of user actions, each one of the plurality of reference queries comprising reference query data and having been submitted by a reference querying user, the corresponding plurality of reference search results for each one of the plurality of reference queries comprising a plurality of reference candidate users displayed in response to the plurality of reference search queries based on profile data of the plurality of reference candidate users stored on a database of an online service, the plurality of user actions comprising actions by the reference querying user directed towards at least one reference candidate user of the plurality of reference search results for the corresponding reference search query, each reaction indication indicating whether the reference candidate user to whom the corresponding user action was directed responded to the corresponding user action with at least one of one or more specified responses;
training, by the computer system, a ranking model using the training data and a loss function, the ranking model comprising a deep learning model and configured to generate similarity scores based on a determined level of similarity between the profile data of the reference candidates users and the reference query data of the reference queries;
receiving, by the computer system, a target query comprising target query data from a computing device of a target querying user;
for each one of a plurality of target candidate users, generating, by the computer system, a corresponding score for a pairing of the one of the plurality of target candidate users and the target query based on a determined level of similarity between profile data of the one of the plurality of target candidate users and the target query data of the target query using the trained ranking model;
and
causing, by the computer system, an indication of at least a portion of the plurality of target candidate users to be displayed on the computing device as search results for the target query based on the generated scores of the plurality of target candidate users.

3 more elements

A system comprising:

at least one hardware processor;
and
a non-transitory machine-readable medium embodying a set of instructions that, when executed by at least one hardware processor, cause the processor to perform operations, the operations comprising:
receiving training data comprising a plurality of reference queries, a plurality of reference search results for each one of the plurality of reference queries;
a plurality of user actions for each one of the plurality of reference queries, and a corresponding reaction indication for each one of the plurality of user actions, each one of the plurality of reference queries comprising reference query data and having been submitted by a reference querying user, the corresponding plurality of reference search results for each one of the plurality of reference queries comprising a plurality of reference candidate users displayed in response to the plurality of reference search queries based on profile data of the plurality of reference candidate users stored on a database of an online service, the plurality of user actions comprising actions by the reference querying user directed towards at least one reference candidate user of the plurality of reference search results for the corresponding reference search query, each reaction indication indicating whether the reference candidate user to whom the corresponding user action was directed responded to the corresponding user action with at least one of one or more specified responses;
training a ranking model using the training data and a loss function, the ranking model comprising a deep learning model and configured to generate similarity scores based on a determined level of similarity between the profile data of the reference candidates users and the reference query data of the reference queries;
receiving a target query comprising target query data from a computing device of a target querying user;
for each one of a plurality of target candidate users, generating a corresponding score for a pairing of the one of the plurality of target candidate users and the target query based on a determined level of similarity between profile data of the one of the plurality of target candidate users and the target query data of the target query using the trained ranking model;
and
causing an indication of at least a portion of the plurality of target candidate users to be displayed on the computing device as search results for the target query based on the generated scores of the plurality of target candidate users.

7 more elements

A non-transitory machine-readable medium embodying a set of instructions that, when executed by at least one hardware processor, cause the processor to perform operations comprising:

receiving training data comprising a plurality of reference queries, a plurality of reference search results for each one of the plurality of reference queries, a plurality of user actions for each one of the plurality of reference queries;
and a corresponding reaction indication for each one of the plurality of user actions, each one of the plurality of reference queries comprising reference query data and having been submitted by a reference querying user, the corresponding plurality of reference search results for each one of the plurality of reference queries comprising a plurality of reference candidate users displayed in response to the plurality of reference search queries based on profile data of the plurality of reference candidate users stored on a database of an online service, the plurality of user actions comprising actions by the reference querying user directed towards at least one reference candidate user of the plurality of reference search results for the corresponding reference search query;
each reaction indication indicating whether the reference candidate user to whom the corresponding user action was directed responded to the corresponding user action with at least one of one or more specified responses;
training a ranking model using the training data and a loss function, the ranking model comprising a deep learning model and configured to generate similarity scores based on a determined level of similarity between the profile data of the reference candidates users and the reference query data of the reference queries;
receiving a target query comprising target query data from a computing device of a target querying user;
for each one of a plurality of target candidate users, generating a corresponding score for a pairing of the one of the plurality of target candidate users and the target query based on a determined level of similarity between profile data of the one of the plurality of target candidate users and the target query data of the target query using the trained ranking model;
and
causing an indication of at least a portion of the plurality of target candidate users to be displayed on the computing device as search results for the target query based on the generated scores of the plurality of target candidate users.

5 more elements

Dependencies are inferred from the claim text.