Systems and methods for unsupervised training in text retrieval tasks
AssigneeSalesforce, Inc.
InventorsRui Meng, Yingbo Zhou, Ye Liu, Semih Yavuz, Ning Yu
Life of this patent
Estimating the term…
Loading patent…
AssigneeSalesforce, Inc.
InventorsRui Meng, Yingbo Zhou, Ye Liu, Semih Yavuz, Ning Yu
Estimating the term…
Embodiments described herein provide systems and methods for training a text retrieval model. A system may generate queries associated with provided documents. The queries may be generated in one or more different manners. Examples of query generation may include extracting relevant spans of text from the documents, prompting a language model for a topic, title, abstractive summary, and/or extractive summary based on the documents. Metadata such as title or other HTML tags may be used as queries. Using the one or more queries, the text retrieval model may be trained using contrastive learning, using the generated query, and positive and negative sample documents. A fine-tuning training phase may be performed using domain-specific data which may also be done with generated query pairs, or may be done in a supervised fashion with provided queries. The text retrieval model may be used to locate documents given an input query.
A method of training a text retrieval model, the method comprising:
A system for training a text retrieval model, the system comprising:
A non-transitory machine-readable medium comprising a plurality of machine-executable instructions which, when executed by one or more processors, are adapted to cause the one or more processors to perform operations comprising:
Dependencies are inferred from the claim text.