Latent question reformulation and information accumulation for multi-hop machine reading

A training system comprising:

• a question answering system comprising:
• a first encoder module configured to receive a question from a computing device via a network, the question including a first plurality of words, and to encode the question into a first vector representation;
• a second encoder module configured to encode a document D into a second vector representation, the document including a second plurality of words;
• a first reading module configured to generate a third vector representation based on the first and second vector representations;
• a first reformulation module configured to generate a first reformulated vector representation for the question based on the first vector representation;
• a second reading module configured to generate a fourth vector representation based on the second vector representation and the first reformulated vector representation;
• a second reformulation module configured to generate a second reformulated vector representation for the question based on first reformulated vector representation;
• and
• an answer module configured to determine an answer to the question based on the second reformulated vector representation and to transmit the answer to the computing device via the network;
• and
• a training module configured to train the question answering system by minimizing a first loss regarding facts support answers to questions, a second loss regarding spans of words, and a third loss regarding classification of answers.

A question answering system comprising:

• a means for receiving a question from a computing device via a network, the question including a first plurality of words, and to encode the question into a first vector representation;
• a means for encoding a document D into a second vector representation, the document including a second plurality of words;
• a means for generating a third vector representation based on the first and second vector representations;
• a means for generating a first reformulated vector representation for the question based on the first vector representation;
• a means for generating a fourth vector representation based on the second vector representation and the first reformulated vector representation;
• a means for generating a second reformulated vector representation for the question based on first reformulated vector representation;
• a means for determining an answer to the question based on the second reformulated vector representation and transmitting the answer to the computing device via the network,
• wherein the means for generating the third vector representation is for determining (a) first and (b) second sets of probabilities that represent (a) a probability of each word of the document being a beginning of an answer span and (b) a probability of each word of the document being an end of the answer span, respectively, the answer span being a span of words of the document that correctly answers the question;
• and
• a means for, based on the first and second sets of probabilities, determining an attention vector of probabilities of entries of the second vector representation being part of the answer span.

A question answering method comprising:

• by one or more processors, receiving a question from a computing device via a network, the question including a first plurality of words, and to encode the question into a first vector representation;
• by the one or more processors, encoding a document D into a second vector representation, the document including a second plurality of words;
• by the one or more processors, generating a third vector representation based on the first and second vector representations;
• by the one or more processors, generating a first reformulated vector representation for the question based on the first vector representation;
• by the one or more processors, generating a fourth vector representation based on the second vector representation and the first reformulated vector representation;
• by the one or more processors, generating a second reformulated vector representation for the question based on first reformulated vector representation;
• by the one or more processors, determining an answer to the question based on the second reformulated vector representation;
• and
• by the one or more processors, transmitting the answer to the computing device via the network,
• by the one or more processors, determining (a) first and (b) second sets of probabilities that represent (a) a probability of each word of the document being a beginning of an answer span and (b) a probability of each word of the document being an end of the answer span, respectively, the answer span being a span of words of the document that correctly answers the question;
• and
• by the one or more processors, based on the first and second sets of probabilities, determining an attention vector of probabilities of entries of the second vector representation being part of the answer span.

A system for answering a question using a document, comprising:

• (A) a plurality of components of sequential hops, wherein a first component hop is coupled in sequence with a second component hop, and wherein:
• (a) the first component hop comprises a first reading module coupled to a first reformulation module,
• (i) the first reading module configured to receive the question and the document and to output to the first reformulation module a first question-aware representation of the document;
• and
• (ii) the first reformation module configured to extract elements from the first question-aware representation of the document and to compute a first updated representation of the question;
• and
• (b) the second component hop comprises a second reading module coupled to a second reformulation module,
• (i) the second reading module configured to receive the first updated representation of the question and the document and to output to the second reformulation module a second question-aware representation of the document;
• and
• (ii) the second reformulation module configured to extract elements from the second question-aware representation of the document and to compute a second updated representation of the question;
• (B) a plurality of parallel heads, each of the plurality of parallel heads including an independent instance of (A) the plurality of components of sequential hops;
• (C) an answering module configured to:
• receive updated representations of the document output by the plurality of parallel heads, respectively;
• and
• based on the updated representations of the document, determine an answer to the question and facts in the document that support the answer to the question,
• wherein the first reading module is configured to determine (a) first and (b) second sets of probabilities that represent (a) a probability of each word of the document being a beginning of an answer span and (b) a probability of each word of the document being an end of the answer span, respectively, the answer span being a span of words of the document that correctly answers the question;
• and
• (D) an extractive attention module configured to, based on the first and second sets of probabilities, determine an attention vector of probabilities of entries of the second question-aware representation being part of the answer span.