Search ranker with cross attention encoder to jointly compute relevance scores of keywords to a query

A computerized method, comprising:

• receiving, via a search engine, a search query from a search engine client communicating with the search engine;
• generating a set of candidate keywords based on the search query;
• jointly computing, via a cross attention encoder, relevance scores of the set of candidate keywords to the search query, and to generate a ranked list of candidate keywords based on the jointly computed relevance scores, wherein the relevance scores are computed by:
• obtaining contextual embeddings for query tokens in the search query and a union of keyword tokens in the set of keywords;
• selectively pooling the contextual embeddings for the query tokens and the union of keyword tokens;
• inputting the selectively pooled contextual embeddings into a classifier that has been trained to predict a relevance score of the plurality of keywords to the search query based on the selectively pooled contextual embeddings, to thereby cause the classifier to output a relevance score for each of query-keyword pairs for the keywords in the set of candidate keywords, wherein the ranked list of candidate keywords is ranked based on the outputted relevance scores;
• and
• outputting the ranked list of candidate keywords;
• and
• generating a search result including content associated with at least one of the candidate keywords in the ranked list, and output the search result to the search engine client, wherein
• the selectively pooled contextual embeddings are input through the classifier to output the relevance scores for the query-keyword pairs for all candidate keywords in a single pass, thereby enabling real-time scoring of multiple query-keyword pairs.

A computing system, comprising:

• a server including processing circuitry and associated memory, the processing circuitry being configured to execute instructions stored in the associated memory to implement:
• a search engine, the search engine being configured to receive a search query from a user via a graphical user interface of a search engine client communicating with the search engine, wherein the search engine includes:
• a retriever module configured to receive the search query and generate a set of candidate keywords based on the search query, from a keyword set that is larger than the candidate keyword set;
• and
• a ranker module configured to jointly compute relevance scores of a set of candidate keywords to the search query, and to generate a ranked list of candidate keywords based on the jointly computed relevance scores, wherein the ranker module includes a cross attention encoder configured to jointly compute the relevance scores by:
• obtaining, via an encoder, contextual embeddings for query tokens in the search query and a union of keyword tokens in the set of keywords;
• selectively pooling the contextual embeddings for the query tokens and the union of keyword tokens;
• inputting the selectively pooled contextual embeddings into a classifier that has been trained to predict a relevance score of the plurality of keywords to the search query based on the selectively pooled contextual embeddings, to thereby cause the classifier to output a relevance score for each of query-keyword pairs for the keywords in the set of candidate keywords, wherein the ranked list of candidate keywords is ranked based on the outputted relevance scores;
• and
• outputting the ranked list of candidate keywords;
• and
• a search result module configured to generate a search result including content associated with at least one of the candidate keywords in the list, and output the search result to the search engine client for display in the graphical user interface, wherein
• selectively pooling the contextual embeddings uses a keyword union attention mask in which each keyword token appears only once, and produces a single embedding for each of the query-keyword pairs for all candidate keywords, and
• the single embeddings of the selectively pooled contextual embeddings are combined and input in a single pass through the classifier to output the relevance scores for the query-keyword pairs for all candidate keywords, thereby enabling real-time scoring of multiple query-keyword pairs.