Generating audio using neural networks

A system comprising:

• one or more computers;
• and
• one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
• receiving data characterizing a sequence of text;
• processing a network input comprising the data characterizing the sequence of text using a neural network to generate a neural network output that defines audio data that is a verbalization of the sequence of text characterized by the network input, wherein:
• the neural network comprises a sequence of one or more convolutional neural network layers;
• and
• each convolutional neural network layer in the sequence of one or more convolutional neural network layers is configured to:
• receive a layer input;
• apply one or more convolution operations to the layer input;
• and
• generate a layer output based at least in part on a result of applying the one or more convolution operations to the layer input;
• wherein the neural network output directly comprises amplitude values of an audio waveform that is the verbalization of the sequence of text characterized by the network input, and
• wherein the amplitude values of the audio waveform are directly generated as an output of an output neural network layer of the neural network, and
• wherein the neural network has been trained by applying a supervised learning technique that depends on (i) ground truth output audio waveforms for each of a set of training examples for the neural network and (ii) corresponding output audio waveforms generated by the neural network.

One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• receiving data characterizing a sequence of text;
• processing a network input comprising the data characterizing the sequence of text using a neural network to generate a neural network output that defines audio data that is a verbalization of the sequence of text characterized by the network input, wherein:
• the neural network comprises a sequence of one or more convolutional neural network layers;
• and
• each convolutional neural network layer in the sequence of one or more convolutional neural network layers is configured to:
• receive a layer input;
• apply one or more convolution operations to the layer input;
• and
• generate a layer output based at least in part on a result of applying the one or more convolution operations to the layer input;
• wherein the neural network output directly comprises amplitude values of an audio waveform that is the verbalization of the sequence of text characterized by the network input,
• wherein the amplitude values of the audio waveform are directly generated as an output of an output neural network layer of the neural network, and
• wherein the neural network has been trained by applying a supervised learning technique that depends on (i) ground truth output audio waveforms for each of a set of training examples for the neural network and (ii) corresponding output audio waveforms generated by the neural network.