Conversational turn analysis neural networks

A system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:

• obtaining unsupervised training data comprising a plurality of dialogue transcripts, each dialogue transcript comprising a sequence of conversational turns;
• training a turn prediction neural network to perform a turn prediction task on the unsupervised training data using unsupervised learning, wherein:
• the turn prediction neural network comprises (i) a turn encoder neural network that is configured to receive an input snippet comprising one or more input conversational turns and to generate an encoded representation of the input snippet in accordance with a set of encoder network parameters and (ii) a turn decoder neural network that is configured to receive the encoded representation of the input snippet and to process the encoded representation to generate a turn prediction, and
• training the turn prediction neural network to perform the turn prediction task comprises training the turn encoder neural network to determine updated values of the encoder network parameters from initial values of the encoder network parameters;
• obtaining supervised training data comprising a plurality of snippets of one or more conversational turns and, for each snippet, a respective target output;
• and
• training a supervised prediction neural network to perform a supervised prediction task on the supervised training data using supervised learning, wherein:
• the supervised prediction neural network comprises (i) the turn encoder neural network and (ii) a prediction neural network that is configured to receive the encoded representation of the input snippet generated by the turn encoder neural network and to process the respective encoded representations to generate a supervised prediction, and
• training the supervised prediction neural network to perform the supervised prediction task comprises training the turn prediction neural network to determine trained values of the encoder network parameters from the updated values of the encoder network parameters that were determined by training the turn prediction neural network on the turn prediction task.

One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

• obtaining unsupervised training data comprising a plurality of dialogue transcripts, each dialogue transcript comprising a sequence of conversational turns;
• training a turn prediction neural network to perform a turn prediction task on the unsupervised training data using unsupervised learning, wherein:
• the turn prediction neural network comprises (i) a turn encoder neural network that is configured to receive an input snippet comprising one or more input conversational turns and to generate an encoded representation of the input snippet in accordance with a set of encoder network parameters and (ii) a turn decoder neural network that is configured to receive the encoded representation of the input snippet and to process the encoded representation to generate a turn prediction, and
• training the turn prediction neural network to perform the turn prediction task comprises training the turn encoder neural network to determine updated values of the encoder network parameters from initial values of the encoder network parameters;
• obtaining supervised training data comprising a plurality of snippets of one or more conversational turns and, for each snippet, a respective target output;
• and
• training a supervised prediction neural network to perform a supervised prediction task on the supervised training data using supervised learning, wherein:
• the supervised prediction neural network comprises (i) the turn encoder neural network and (ii) a prediction neural network that is configured to receive the encoded representation of the input snippet generated by the turn encoder neural network and to process the respective encoded representations to generate a supervised prediction, and
• training the supervised prediction neural network to perform the supervised prediction task comprises training the turn prediction neural network to determine trained values of the encoder network parameters from the updated values of the encoder network parameters that were determined by training the turn prediction neural network on the turn prediction task.