Systems and methods for parallel wave generation in end-to-end text-to-speech

A computer-implemented method for training an end-to-end text-to-speech system to synthesize speech from an input text, comprising:

• encoding, via an encoder comprising one or more convolutional blocks, the input text into encoder hidden representations including key representations and value representations;
• autoregressively decoding, using a decoder, the encoder hidden representations with attention into frame-level hidden representations;
• processing, using a convolutional processing block, the frame-level hidden representations into sample-level hidden representations;
• generating, using an autoregressive vocoder, synthesized waveforms corresponding to the input text conditioned on the sample-level hidden representations;
• and
• distilling the autoregressive vocoder to obtain a distilled parallel vocoder based on Gaussian inverse autoregressive flow (IAF) using at least a regularized Kullback-Leibler (KL) divergence between output distributions of the autoregressive vocoder and the distilled parallel vocoder.

A computer-implemented method for training a text-to-speech system to synthesize speech from ground-truth spectrograms, comprising:

• receiving, at an autoregressive vocoder, ground-truth spectrograms for waveform synthesizing;
• receiving, at a parallel vocoder distilled from the autoregressive vocoder based on Gaussian inverse autoregressive flow (IAF), the ground-truth spectrograms for waveform synthesizing;
• and
• training the parallel vocoder using a loss function having a linear combination of a frame-level loss and a regularized Kullback-Leibler (KL) divergence between waveform distributions of the autoregressive vocoder and the parallel vocoder.