Back to results
US 12,585,661 B1Granted
User engagement quality estimation
AssigneeNotion Labs, Inc.
InventorsMuhua Zhou, Zachary David Smith, Jiahui Jiang
Life of this patent
Estimating the term…
Abstract
The present disclosure relates to systems and methods for improved searching. In some implementations, the approaches herein can be used to rank search results to provide more relevant results to users, for example by monitoring the interactions of one or more users with pages or other items. In some implementations, the approaches herein can be used to refine and improve the performance of a search result ranking model over time.
Claims20
A computer-implemented method for determining user engagement quality, the computer-implemented method comprising:
- receiving a search request comprising a search string from a user;
- generating a retrieval query based at least in part on the search string;
- providing the retrieval query to a search engine;
- receiving a set of search results from the search engine, each search result corresponding to a page;
- ranking search results of the set of search results using a ranking model to generate a set of ranked search results,
- wherein the ranking model is configured to rank the search results of the set of search results based at least in part on an engagement score, wherein the engagement score is based on at least three of:
- dwell time, page edit, page comment, scrolling, back navigation, forward navigation, or subsequent searching,
- wherein dwell time indicates a length of time a page associated with a selected search result was viewed,
- wherein page edit indicates if the page was edited,
- wherein page comment indicates whether a comment was left on the page,
- wherein scrolling indicates whether a viewer of the page scrolled the page,
- wherein back navigation indicates if a user navigated back to the set of ranked search results,
- wherein forward navigation indicates whether a user navigated to another page, and
- wherein subsequent searching indicates whether a user performed another search after viewing the selected search result;
- providing the set of ranked search results to the user;
- receiving a request from the user for a page included in the set of ranked search results;
- providing, in response to the request, the page to the user;
- monitoring user interaction information, wherein the user interaction information comprises at least three of:
- dwell time, page edit, page comment, scrolling, back navigation, forward navigation, or subsequent searching;
- determining, based on at least a portion of the user interaction information, an updated engagement score;
- and
- storing, in a non-volatile computer-readable storage medium, the updated engagement score,
- wherein the engagement score is associated with the page,
- wherein the engagement score is associated with the search string, and
- wherein the ranking model is configured to be trained based at least in part on the engagement score.
22 more elements
A computer-implemented method for determining user engagement quality, the computer-implemented method comprising:
- accessing a set of search results responsive to a search requested by a user,
- wherein each search result of the set of search results corresponds to a page,
- wherein the search results are determined by executing a retrieval query based at least in part on a search string submitted by the user;
- ranking each search result of the set of search results using a ranking model to generate a set of rank-ordered search results,
- wherein the ranking model is configured to rank the search results based at least in part on an engagement score determined based on first user interaction data indicating user interaction with each corresponding page,
- wherein the first user interaction data is based on one or more of:
- page edit or page comment;
- providing the set of rank-ordered search results to the user;
- receiving a request from the user for a page included in the set of rank-ordered search results;
- providing, in response to the request, the page to the user;
- monitoring user interaction with the page to collect second user interaction data;
- and
- determining, based on the second user interaction data, a second engagement score,
- wherein the engagement score and the second engagement score are associated with the page,
- wherein the engagement score and the second engagement score are associated with a search string included in the search requested by the user, and
- wherein the ranking model is configured to be trained based at least in part on the engagement score.
13 more elements
A system for determining user engagement quality comprising:
- at least one processor;
- and
- a non-volatile, computer-readable storage medium having instructions stored thereon that, when executed by the at least one processor, cause the system to:
- accessing a set of search results responsive to a search requested by a user,
- wherein each search result of the set of search results corresponds to a page,
- wherein the search results are determined by executing a retrieval query based at least in part on a search string submitted by the user;
- ranking each search result of the set of search results using a ranking model to generate a set of rank-ordered search results,
- wherein the ranking model is configured to rank the search results based at least in part on an engagement score determined based on first user interaction data indicating user interaction with each corresponding page,
- wherein the first user interaction data is based on one or more of:
- page edit or page comment;
- providing the set of rank-ordered search results to the user;
- receiving a request from the user for a page included in the set of rank-ordered search results;
- providing, in response to the request, the page to the user;
- monitoring user interaction with the page to collect second user interaction data;
- and
- determining, based on the second user interaction data, a second engagement score,
- wherein the engagement score and second engagement score are associated with the page,
- wherein the engagement score and second engagement score associated with a search string included in the search requested by the user, and
- wherein the ranking model is configured to be trained based at least in part on the engagement score.
16 more elements
Dependencies are inferred from the claim text.