Method and system for generating longform technical question and answer dataset

A system comprising:

• at least one memory storing programmed instructions;
• one or more Input/Output (I/O) interfaces;
• and
• one or more hardware processors operatively coupled to the at least one memory, wherein the one or more hardware processors are configured by the programmed instructions to:
• receive a technical textbook document comprising a plurality of sections;
• extract a plurality of contexts associated with each of the plurality of sections using a rule based preprocessing technique, wherein the rule based preprocessing technique removes a plurality of document sentence structures including figures, tables, footnotes and page numbers from the technical textbook document;
• generate a first plurality of questions by a neural network machine learning (ML) model, based on the plurality of contexts associated with each of the plurality of sections, wherein randomness in an optimized autoregressive model is excluded by maintaining predefined temperature values, wherein when a temperature value is set to “zero”, the optimized autoregressive model generates same set of questions multiple times, wherein when the temperature value is set to “one”, the optimized autoregressive model randomly generates varied set of questions every time, wherein the predefined temperature value is “zero” to avoid randomness in question generation;
• simultaneously generate a plurality of answerable questions based on the plurality of contexts associated with each of the plurality of sections;
• generate a combined plurality of questions by combining the first plurality of questions and the plurality of answerable questions, wherein combining is performed using one of a) concatenation and b) join operation;
• generate an answer corresponding to each of the combined plurality of questions, wherein a plurality of sentences associated with a generated answer are one of a) contiguous and b) non-contiguous in the technical textbook document;
• compute a mapping score for each of a plurality of answers based on a comparison between each of the plurality of sentences associated with each of the plurality of answers and the corresponding context by comparing each sentence ain the generated answer with each sentence cj in the context and calculating the mapping score Sij=match (ai, cj), wherein the mapping score sij depends on a ratio of number of matched tokens and a total number of tokens in an answer sentence;
• select a plurality of optimal answers from the plurality of answers based on a corresponding mapping score and a predefined mapping threshold, wherein the plurality of answers having the corresponding mapping score greater than the predefined mapping threshold are selected, wherein when the mapping score Sij is greater than the predefined mapping threshold T, then the cj is included in a map-backed answer list and marked that it is not considered for matching for subsequent answer sentences,
• wherein when multiple context sentences sij>T against an answer sentence di, then the context sentence cj having maximum score is selected in the map-backed answer list,
• wherein when the map-backed answer list is empty for a question, then that question is not answerable from the context and that question is excluded from question and answer dataset, further achieves (1) map-back the generated answers by the optimized autoregressive model into an extracted answer, and (2) answerability of generated questions by the optimized autoregressive model;
• generate a longform technical question and answer dataset based on the combined plurality of questions and an optimal answer corresponding to each of the combined plurality of questions, wherein the generated optimal question and answer dataset based on a plurality of evaluation metrics, wherein the plurality of evaluation metrics comprises the long-formness, a question completeness, a question complexity, a complexity diversity, a concept coverage diversity, and a sentence level precision,
• wherein the long-formness checks whether each of the plurality of optimal answers comprises at least two sentences,
• wherein the question completeness checks whether the plurality of answerable questions does not have any unresolved co-reference and indefinite words,
• wherein the question complexity identifies whether at least one concept is available in each of the plurality of answerable questions,
• wherein the complexity diversity checks whether the plurality of answerable questions are with varying complexity range comprising easy, medium and hard;
• and
• training a model with the generated longform technical question and answer dataset for answer generation and evaluating the trained model with (i) a sentence level evaluation to determine percentage of answer sentences predicted out of all answer sentences, and (ii) a token level evaluation performed using a n-gram matching based f-score calculation between a predicted answer and an actual answer.

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

• receiving a technical textbook document comprising a plurality of sections;
• extracting a plurality of contexts associated with each of the plurality of sections using a rule based preprocessing technique, wherein the rule based preprocessing technique removes a plurality of document sentence structures including figures, tables, footnotes and page numbers from the technical textbook document;
• generating a first plurality of questions by a neural network machine learning (ML) model, based on the plurality of contexts associated with each of the plurality of sections, wherein randomness in an optimized autoregressive model is excluded by maintaining predefined temperature values, wherein when a temperature value is set to “zero”, the optimized autoregressive model generates same set of questions multiple times, wherein when the temperature value is set to “one”, the optimized autoregressive model randomly generates varied set of questions every time, wherein the predefined temperature value is “zero” to avoid randomness in question generation;
• simultaneously generating a plurality of answerable questions based on the plurality of contexts associated with each of the plurality of sections;
• generating a combined plurality of questions by combining the first plurality of questions and the plurality of answerable questions, wherein combining is performed using one of a) concatenation and b) join operation;
• generating an answer corresponding to each of the combined plurality of questions, wherein a plurality of sentences associated with a generated answer are one of a) contiguous and b) non-contiguous in the technical textbook document;
• computing a mapping score for each of a plurality of answers based on a comparison between each of the plurality of sentences associated with each of the plurality of answers and the corresponding context by comparing each sentence αi in the generated answer with each sentence cj in the context and calculating the mapping score sij=match (ai, Cj), wherein the mapping score sij depends on a ratio of number of matched tokens and a total number of tokens in an answer sentence;
• selecting a plurality of optimal answers from the plurality of answers based on a corresponding mapping score and a predefined mapping threshold, wherein the plurality of answers having the corresponding mapping score greater than the predefined mapping threshold are selected, wherein when the mapping score Sij is greater than the predefined mapping threshold T, then the cj is included in a map-backed answer list and marked that it is not considered for matching for subsequent answer sentences,
• wherein when multiple context sentences Sij>T against an answer sentence αi, then the context sentence cj having maximum score is selected in the map-backed answer list,
• wherein when the map-backed answer list is empty for a question, then that question is not answerable from the context and that question is excluded from question and answer dataset, further achieves (1) map-back the generated answers by the optimized autoregressive model into an extracted answer, and (2) answerability of generated questions by the optimized autoregressive model;
• generating a longform technical question and answer dataset based on the combined plurality of questions and an optimal answer corresponding to each of the combined plurality of questions, wherein the generated optimal question and answer dataset based on a plurality of evaluation metrics, wherein the plurality of evaluation metrics comprises the long-formness, a question completeness, a question complexity, a complexity diversity, a concept coverage diversity, and a sentence level precision,
• wherein the long-formness checks whether each of the plurality of optimal answers comprises at least two sentences,
• wherein the question completeness checks whether the plurality of answerable questions does not have any unresolved co-reference and indefinite words,
• wherein the question complexity identifies whether at least one concept is available in each of the plurality of answerable questions,
• wherein the complexity diversity checks whether the plurality of answerable questions are with varying complexity range comprising easy, medium and hard;
• and
• training a model with the generated longform technical question and answer dataset for answer generation and evaluating the trained model with (i) a sentence level evaluation to determine percentage of answer sentences predicted out of all answer sentences, and (ii) a token level evaluation performed using a n-gram matching based f-score calculation between a predicted answer and an actual answer.