Learning language representation with logical inductive bias
AssigneeTencent America LLC
InventorJianshu CHEN
Life of this patent
Estimating the term…
Abstract
A method including receiving input comprising natural language texts; pre-training a First-Order Logic Network (FOLNet) neural network model on unlabeled texts included in the natural language texts, the FOLNet neural network model comprising of a plurality of layers; processing the input through the plurality of layers of the FOLNet neural network model; encoding a logical inductive bias using the FOLNet neural network model; outputting one or more tensors based on the logical inductive bias; and predicting an outcome using the one or more tensors.
Claims20
A method executed by at least one processor, the method comprising:
- receiving input comprising natural language texts;
- pre-training a First-Order Logic Network (FOLNet) neural network model on unlabeled texts included in the natural language texts, the FOLNet neural network model comprising of a plurality of layers;
- processing the input through the plurality of layers of the FOLNet neural network model;
- encoding a logical inductive bias using the FOLNet neural network model;
- outputting one or more tensors based on the logical inductive bias;
- and
- predicting an outcome using the one or more tensors.
An apparatus comprising:
- at least one memory configured to store program code;
- and
- at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising:
- receiving code configured to cause the at least one processor to receive input comprising natural language texts;
- pre-training code configured to cause the at least one processor to pre-train a First-Order Logic Network (FOLNet) neural network model on unlabeled texts included in the natural language texts, the FOLNet neural network model comprising of a plurality of layers;
- processing code configured to cause the at least one processor to process the input through the plurality of layers of the FOLNet neural network model;
- encoding code configured to cause the at least one processor to encode a logical inductive bias using the FOLNet neural network model;
- outputting code configured to cause the at least one processor to output one or more tensors based on the logical inductive bias;
- and
- predicting code configured to cause the at least one processor to predict an outcome using the one or more tensors.
A non-transitory computer-readable storage medium, storing instructions, which, when executed by at least one processor, cause the at least one processor to:
- receive input comprising natural language texts;
- pre-train a First-Order Logic Network (FOLNet) neural network model on unlabeled texts included in the natural language texts, the FOLNet neural network model comprising of a plurality of layers;
- process the input through the plurality of layers of the FOLNet neural network model;
- encode a logical inductive bias using the FOLNet neural network model;
- output one or more tensors based on the logical inductive bias;
- and
- predict an outcome using the one or more tensors.
Dependencies are inferred from the claim text.