Guided dialogue using language generation neural networks and search

A method, implemented by one or more computers, of training a neural network system to enable use of an agent comprising a first language generation neural network to obtain information by dialogue between the user and the agent, the method comprising:

• determining a context input for a current dialogue iteration and, at one or more dialogue update iterations:
• obtaining a natural language output statement from an action selection policy neural network comprising the first language generation neural network, by generating a natural language token for the natural language output statement at each of a succession of time steps until the end of a statement generation episode, wherein generating the token at a time step comprises:
• processing the context input for the current dialogue iteration and tokens previously generated during the episode, using the action selection policy neural network, to select an action, wherein the action is to select a next token for the natural language output statement;
• processing at least a portion of the context input and the natural language output statement using a response selection neural network to determine a first reward for the natural language output statement;
• processing at least a portion of the context input and the natural language output statement using a rule violation detection neural network to determine a second reward for the natural language output statement;
• and
• training the action selection policy neural network comprising the first language generation neural network, based on the first reward and the second reward, using a reinforcement learning technique.

A system comprising:

• one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
• determining a context input for a current dialogue iteration and, at one or more dialogue update iterations:
• obtaining a natural language output statement from an action selection policy neural network comprising the first language generation neural network, by generating a natural language token for the natural language output statement at each of a succession of time steps until the end of a statement generation episode, wherein generating the token at a time step comprises:
• processing the context input for the current dialogue iteration and tokens previously generated during the episode, using the action selection policy neural network, to select an action, wherein the action is to select a next token for the natural language output statement;
• processing at least a portion of the context input and the natural language output statement using a response selection neural network to determine a first reward for the natural language output statement;
• processing at least a portion of the context input and the natural language output statement using a rule violation detection neural network to determine a second reward for the natural language output statement;
• and
• training the action selection policy neural network comprising the first language generation neural network, based on the first reward and the second reward, using a reinforcement learning technique.

One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• determining a context input for a current dialogue iteration and, at one or more dialogue update iterations:
• obtaining a natural language output statement from an action selection policy neural network comprising the first language generation neural network, by generating a natural language token for the natural language output statement at each of a succession of time steps until the end of a statement generation episode, wherein generating the token at a time step comprises:
• processing the context input for the current dialogue iteration and tokens previously generated during the episode, using the action selection policy neural network, to select an action, wherein the action is to select a next token for the natural language output statement;
• processing at least a portion of the context input and the natural language output statement using a response selection neural network to determine a first reward for the natural language output statement;
• processing at least a portion of the context input and the natural language output statement using a rule violation detection neural network to determine a second reward for the natural language output statement;
• and
• training the action selection policy neural network comprising the first language generation neural network, based on the first reward and the second reward, using a reinforcement learning technique.