Selection-inference neural network systems

A method performed by one or more computers, the method comprising:

• obtaining a context input comprising context information, the context information comprising one or more natural language statements;
• receiving a query input;
• generating a response to the query input by performing update iterations using one or more neural networks until a termination criterion is satisfied, the generating comprising, at each update iteration:
• generating a selection input comprising the context input, the query input, and a first prompt;
• processing the selection input using at least one of the one or more neural networks to generate a selection output comprising a subset of the natural language statements from the context input;
• generating an inference input comprising the selection output and a second, different prompt;
• processing the inference input using at least one of the one or more neural networks to generate an inference output comprising a natural language statement representing a new fact;
• and
• updating the context input to include the natural language statement representing the new fact;
• and
• providing a natural language output derived from the inference output of a last update iteration.

A method performed by one or more computers, the method comprising:

• obtaining a context input comprising context information, the context information comprising one or more natural language statements;
• receiving a query input;
• generating a response to the query input by performing update iterations using one or more neural networks until a termination criterion is satisfied, the generating comprising, at each update iteration:
• processing a selection input comprising the context input and the query input using at least one of the one or more neural networks to generate a selection output comprising one or more of the natural language statements from the context input, wherein processing the selection input comprises:
• generating, by the at least one of the one or more neural networks, tokens using constrained sampling, wherein the tokens comprise a valid continuation of a natural language statement in the context input;
• and
• generating the selection output using the tokens;
• processing an inference input comprising the selection output using at least one of the one or more neural networks to generate an inference output that comprises a natural language statement representing a new fact;
• and
• updating the context input to include the natural language statement representing the new fact;
• and
• providing a natural language output derived from the inference output of a last update iteration.

A method performed by one or more computers, the method comprising:

• obtaining a context input comprising context information, the context information comprising one or more natural language statements;
• receiving a query input;
• generating a response to the query input by performing update iterations using one or more neural networks, the generating comprising, at each update iteration:
• processing a selection input comprising the context input and the query input using at least one of the one or more neural networks to generate a selection output comprising one or more of the natural language statements from the context input;
• processing an inference input comprising the selection output using at least one of the one or more neural networks to generate an inference output that comprises a natural language statement representing a new fact;
• updating the context input to include the natural language statement representing the new fact;
• processing a halting input comprising the query input and the inference output using at least one of the one or more neural networks to generate an indication of whether the inference output is a valid response to the query input;
• and
• terminating the update iterations when the indication generated by the at least one of the one or more neural networks satisfies a termination criterion;
• and
• providing a natural language output derived from the inference output of a last update iteration.