Directly identifying items from an item catalog satisfying a received query using a model determining measures of similarity between items in the item catalog and the query

A method comprising:

• maintaining, at an online system, a database of content items, the database identifying each content item and values of one or more attributes of each content items;
• creating one or more templates for natural language descriptions of attributes for each content item of the database, each including a content item identifier of a content item, a description of an attribute, a value of the attribute for the content item, and natural language text;
• generating a training set including one or more examples comprising natural language descriptions of content items of the database and values of one or more attributes for the content item from the one or more templates and the item catalog, each example corresponding to the content item and including a plurality of tokens in positions, with values of one or more tokens determined from values of one or more attributes of the content item;
• training a corpus model to receive a natural language description of the content item and to output one or more embeddings in a vector space for one or more tokens in the natural language description of the content item by:
• applying the corpus model to each example of the training set and backpropagating one or more error terms based on a difference between a predicted token generated by the corpus model for a position of an example and a token included at the position of the example until one or more loss functions satisfy one or more criteria;
• obtaining selection training data from prior searches for content items obtained by the online system, the selection training data comprising a plurality of selection examples, a selection example including a query term and a plurality of pairs that each include a content item identifier and an affinity score between a content item corresponding to the item identifier and the query term;
• training a model comprising the corpus model and a mapping layer that receives an embedding from the corpus model and outputs a predicted similarity of the embedding to content item embeddings for each content item of the database by:
• applying the model to each selection example of the training set and backpropagating one or more error terms based on a difference between a predicted similarity between the embedding from the corpus model and a content item embedding and the affinity score between the query term of the selection example and the content item embedding until one or more loss functions satisfy one or more criteria;
• and
• storing parameters comprising the trained model on a computer readable storage medium.

A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:

• obtain, at the online concierge system, an item catalog for one or more warehouses, the item catalog for a warehouse identifying items offered by the warehouse and attributes of each item offered by the warehouse;
• create one or more templates for natural language descriptions of attributes for each item of the item catalog, each including the item identifier of an item, a description of an attribute, a value of the attribute for the item, and natural language text;
• generate a training set including one or more examples comprising natural language descriptions of items of the item catalog and values of one or more attributes for the item from the one or more templates and the item catalog, each example corresponding to the item and including a plurality of tokens in positions, with values of one or more tokens determined from values of one or more attributes of the item;
• train a corpus model to receive a natural language description of the item and to output one or more embeddings in a vector space for one or more tokens in the natural language description of the item by:
• applying the corpus model to each example of the training set and backpropagating one or more error terms based on a difference between a predicted token generated by the corpus model for a position of an example and a token included at the position of the example until one or more loss functions satisfy one or more criteria;
• obtain selection training data from prior searches for items obtained by the online concierge system, the selection training data comprising a plurality of selection examples, a selection example including a query term and a plurality of pairs that each include an item identifier and an affinity score between an item corresponding to the item identifier and the query term;
• train a model comprising the corpus model and a mapping layer that receives an embedding from the corpus model and outputs a predicted similarity of the embedding to item embeddings for each item of the item catalog by:
• applying the model to each selection example of the training set and backpropagating one or more error terms based on a difference between a predicted similarity between the embedding from the corpus model and an item embedding and the affinity score between the query term of the selection example and the item embedding until one or more loss functions satisfy one or more criteria;
• and
• store parameters comprising the trained model on a computer readable storage medium.