Multitask learning as question answering

A non-transitory machine-readable medium comprising executable code which when executed by one or more processors associated with a computing device are adapted to cause the one or more processors to perform a method comprising:

• encoding first words from a context and second words from a question, wherein the question is separate from but related to the context, the encodings performed in parallel using a multi-layer encoder;
• decoding the encoded context and the encoded question using a multi-layer decoder;
• generating, based on an output from the decoder, a first distribution over the first words from the context, a second distribution over the second words from the question, and a third distribution over third words in a generative vocabulary that comprises a source of words;
• generating a first weight of the first distribution, a second weight of the second distribution, and a third weight of the third distribution
• generating a composite distribution based on the first weight, second weight, and third weight;
• and
• selecting words for inclusion in an answer using the composite distribution;
• and
• training the multi-layer encoder and the multi-layer decoder against a subset of task types, wherein the subset of task types are selected according to an anti-curriculum strategy.

A system for natural language processing, the system comprising:

• a memory storing instructions;
• and
• a processor coupled with the memory and configured, when executing the instructions on the memory, to cause the system to:
• encode first words from a context and second words from a question, wherein the question is separate from but related to the context, the encodings performed in parallel;
• decode the encoded context and the encoded question;
• generate, based on the decoded context and the decoded question, a first distribution over the first words from the context, a second distribution over the second words from the question, and a third distribution over third words in a generative vocabulary that comprises a source of words;
• generate a first weight of the first distribution, a second weight of the second distribution, and a third weight of the third distribution;
• generate a composite distribution based on the first weight, second weight, and third weight;
• and
• select words for inclusion in an answer using the composite distribution,
• wherein the system is trained against a subset of task types, and
• wherein the subset of task types are selected according to an anti-curriculum strategy.