Multilingual Batch Alignment Training For Language Models
AssigneeOracle International Corporation
InventorsAmit Agarwal, Tao Sheng, Hansa Meghwani
Life of this patent
Estimating the term…
Loading patent…
AssigneeOracle International Corporation
InventorsAmit Agarwal, Tao Sheng, Hansa Meghwani
Estimating the term…
Techniques for multilingual batch alignment training and/or fine-tuning for language models are included herein. The system generates and/or accesses data points from batches of data having semantically equivalent content across languages and/or semantically equivalent prompts across languages. The prompts include queries related to a topic or a predictive language task. The data points include one or more sets of resources that are semantic equivalents in different languages. Data points for which a resource enables a correct response are used as positive training data points for reinforcement learning. Data points for which a resource does not enable a correct response are considered as negative training data points for reinforcement learning. The data points are used for alignment training and/or other reinforcement learning for generative language models.
A method comprising:
One or more non-transitory computer readable media comprising instructions which, when executed by one or more hardware processors, cause performance of operations comprising:
A system, comprising:
Dependencies are inferred from the claim text.