Dynamic memory network

A processor-based device including:

• a processor and memory coupled to the processor that implement:
• a question module configured to compute a representation of a question, the question module receiving a question posed against one or more input facts stored as vector representations of input facts;
• at least one input fact memory, in an input encoding module, that stores a vector representation of input facts from which questions are answered and an end-of-passes representation;
• at least one question memory that stores a vector representation of the question at hand;
• a plurality of episodic memories to which attention gate weights are assigned;
• gate attention means for:
• in a first pass traversing the at least one input fact memory, applying a gating function to the input facts and contents received directly from the question memory to calculate updated attention gate weights by iterating the input facts from the at least one input fact memory, and
• in one or more subsequent passes each re-traversing the at least one input fact memory, applying the gating function to the input facts and contents received directly from the question memory to re-calculate updated attention gate weights from an episodic memory that has been updated from a most recent previous pass;
• wherein the gate attention means assigns the calculated attention gate weights for a current pass to a current episodic memory and particular attention calculated gate weights indicate relevance of particular input facts;
• updating a final hidden state of a gated recurrent unit (GRU) based on the re-calculated attention gate weights, and
• computing an updated episodic memory vector for the respective episodic memory based on the re-calculated attention gate weights and a contextual vector generated by the updated final hidden state of the GRU during the current pass;
• and
• completing the one or more subsequent passes when the end-of-passes representation is chosen as a most relevant fact during the respective pass;
• and
• an output that upon completion of the one or more subsequent passes, sends a final outcome from the most recent episodic memory in the input encoding module to an answer module that is separate from the input encoding module and that iteratively generates a sequence of one or more output labels responsive to the question.

A method comprising:

• receiving, at a question module configured to compute a representation of a question, a question posed against one or more input facts stored as vector representations of input facts at a plurality of input fact memories in an input encoding module, wherein the plurality of input fact memories are sequentially connected and at least one input fact memory stores an end-of-passes representation;
• storing a vector representation of the question at a question memory;
• in a first pass sequentially traversing the plurality of input fact memories:
• applying a gating function to the input facts in the one or a plurality of ore input fact memories and a current copy of the vector representation of the question in the question memory,
• calculating, based on application of the gating function, attention gate weights by iterating the input facts from the a plurality of input fact memories in the first pass, and
• assigning the attention gate weights to a first episodic memory in the input encoding module;
• in one or more subsequent passes each sequentially re-traversing the plurality of input fact memories:
• re-applying the gating function to the input facts and updated attention gate weights from an episodic memory that has been updated from a most recent previous pass,
• re-calculating new attention gate weights by re-iterating the input facts from the a plurality of input fact memories in each respective pass, and
• assigning the newly re-calculated attention gate weights to a most recent episodic memory in the input encoding module;
• updating a final hidden state of a gated recurrent unit (GRU) based on the newly re-calculated attention gate weight, and
• computing an updated episodic memory vector for the respective episodic memory based on the newly re-calculated attention gate weights and a contextual vector generated by the updated final hidden state of the GRU during the respective pass;
• and
• completing the one or more subsequent passes when the end-of-passes representation is chosen as a most relevant fact during the respective pass;
• and
• upon completion of the one or more subsequent passes, outputting a final outcome from the most recent episodic memory in the input encoding module to an answer module that is separate from the input encoding module.