Loading patent…
AssigneeOracle International Corporation
InventorsSwetasudha Panda, Ariel Kobren, Michael Louis Wick, Qinlan Shen
Estimating the term…
Bias in a language model generated through fine tuning of a pre-trained language model may be mitigated, whether the bias may be incorporated in the pre-trained language model or in fine-tuning data. A pre-trained language model may be fine-tuned using downstream training data. Prior to tuning, elements within the downstream data may be identified that either match or serve as proxies for one or more identity elements associated with training bias sensitivity. Proxy elements may be identified using an analysis of distributions of the downstream elements and distributions of identity elements. Once the elements are identified, instances of the identified elements may be replaced in the downstream data with one or more masking element to generate masked downstream data. A fine-tuned language model with reduced bias may then be generated from the pre-trained language model by tuning the pre-trained language model using the masked downstream data.
A method, comprising:
One or more non-transitory, computer-readable storage media storing program instructions that, when executed on or across one or more computing devices, cause the one or more computing devices to implement:
A system, comprising:
Dependencies are inferred from the claim text.