Systems and methods for factual extraction from language model

A system for extracting factual information from a language model, the system comprising:

• a communication interface for receiving a query for factual information;
• a memory storing a plurality of machine readable instructions;
• and
• a processor reading and executing the instructions from the memory to perform operations comprising:
• receive, via the communication interface, a query for factual information;
• transform the query into a natural language prompt in a format of a query subject and a queried relation;
• encode, via an embedding layer of a pre-trained language model, the natural language prompt into a first embedding;
• encode, via the adapter model, the first embedding into a second embedding based on a probability that the second embedding returns the factual information when the second embedding is fed the first attention layer of the pre-trained language model;
• decode, by the first attention layer of the pre-trained language mode, the second embedding into a response to the query;
• and
• extract the factual information from the decoded response to the query.

A processor-readable non-transitory storage medium storing a plurality of processor-executable instructions for extracting factual information from a language model, the instructions being executed by one or more processors to perform operations comprising:

• receiving, via a communication interface, a query for factual information;
• transforming the query into a natural language prompt in a format of a query subject and a queried relation;
• encoding, via an embedding layer of a pre-trained language model, the natural language prompt into a first embedding;
• encoding, via an adapter model, the first embedding into a second embedding based on a probability that the second embedding returns the factual information when the second embedding is fed the first attention layer of the pre-trained language model, wherein the adapter model is placed between the embedding layer of the pre-trained language model and the first embedding layer of the pre-trained language model;
• decoding, by the first attention layer of the pre-trained language mode, the second embedding into a response to the query;
• and
• extracting the factual information from the decoded response to the query.