Synthesizing speech from text using neural networks

A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

• generating, from an input character sequence, an output sequence of audio data representing the input character sequence, wherein the output sequence of audio data comprises a respective audio output sample for each of a plurality of time steps, including, for each of the plurality of time steps:
• generating a mel-frequency spectrogram for the time step by processing a representation of a respective portion of the input character sequence using a decoder neural network;
• generating a probability distribution over a plurality of possible audio output samples for the time step by processing the mel-frequency spectrogram for the time step using a vocoder neural network;
• and
• selecting the audio output sample for the time step from the plurality of possible audio output samples in accordance with the probability distribution.

One or more non-transitory computer-readable storage media storing instructions that are operable, when executed by one or more computers, to cause the one or more computers to perform operations comprising:

• generating, from an input character sequence, an output sequence of audio data representing the input character sequence, wherein the output sequence of audio data comprises a respective audio output sample for each of a plurality of time steps, including, for each of the plurality of time steps:
• generating a mel-frequency spectrogram for the time step by processing a representation of a respective portion of the input character sequence using a decoder neural network;
• generating a probability distribution over a plurality of possible audio output samples for the time step by processing the mel-frequency spectrogram for the time step using a vocoder neural network;
• and
• selecting the audio output sample for the time step from the plurality of possible audio output samples in accordance with the probability distribution.