Self-supervision in table question answering

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computing device to cause the computing device to:

• obtain a table comprising a plurality of entries, wherein each entry corresponds to a particular column and particular row of the table;
• identify one or more of the entries in the table that correspond to a target answer of a natural language query;
• generate an intermediate representation of the table comprising the rows corresponding to the identified one or more entries, wherein the intermediate representation masks each of the identified one or more entries;
• generate, for at least the masked entries in the intermediate representation, a set of natural language question and answer pairs, wherein generating a first natural language question and answer pair in the set of natural language question and answer pairs comprises:
• generating a combined embedding for a first masked entry by combining a plurality of distinct embedding types, wherein the distinct embedding types comprise at least token embeddings, column embeddings, and row embeddings for a first row associated with the first masked entry;
• generate a first natural language question for a first one of the masked entries, that, when answered using the table, results in the first masked entry being an answer to the first natural language question, wherein an encoder-based transformer layer of a machine learning framework generates contextual information from the combined embedding, and wherein the first natural language question is generated by a generation layer of the machine learning framework using the contextual information as input;
• determine a vocabulary for at least one domain based at least in part on the set of natural language question and answer pairs and feedback responsive to the generated set of natural language question and answer pairs;
• regenerate the set of natural language question and answer pairs based on the determined vocabulary;
• and
• use the regenerated set of natural language question and answer pairs to train a machine learning model.

A system comprising:

• a memory configured to store program instructions;
• and
• a processor operatively coupled to the memory to execute the program instructions to:
• obtain a table comprising a plurality of entries, wherein each entry corresponds to a particular column and particular row of the table;
• identify one or more of the entries in the table that correspond to a target answer of a natural language query;
• generate, for at least the masked entries in the intermediate representation, a set of natural language question and answer pairs, wherein generating a first natural language question and answer pair in the set of natural language question and answer pairs comprises:
• generating a combined embedding for a first masked entry by combining a plurality of distinct embedding types, wherein the distinct embedding types comprise at least token embeddings, column embeddings, and row embeddings for a first row associated with the first masked entry;
• generate a first natural language question for a first one of the masked entries, that, when answered using the table, results in the first masked entry being an answer to the first natural language question, wherein an encoder-based transformer layer of a machine learning framework generates contextual information from the combined embedding, and wherein the first natural language question is generated by a generation layer of the machine learning framework using the contextual information as input;
• determine a vocabulary for at least one domain based at least in part on the set of natural language question and answer pairs and feedback responsive to the generated set of natural language question and answer pairs;
• regenerate the set of natural language question and answer pairs based on the determined vocabulary;
• and
• use the regenerated set of natural language question and answer pairs to train a machine learning model.