Real-time speaker-dependent neural vocoder
AssigneesAdobe Inc., The Trustees of Princeton University
InventorsZeyu Jin, Gautham J. Mysore, Jingwan Lu, Adam Finkelstein
Life of this patent
Estimating the term…
Abstract
Techniques for a recursive deep-learning approach for performing speech synthesis using a repeatable structure that splits an input tensor into a left half and right half similar to the operation of the Fast Fourier Transform, performs a 1-D convolution on each respective half, performs a summation and then applies a post-processing function. The repeatable structure may be utilized in a series configuration to operate as a vocoder or perform other speech processing functions.
Claims20
A method for generating speech samples, the method comprising:
- receiving an input tensor;
- splitting said received input tensor into a first portion and a second portion;
- performing a 1×1 convolution respectively on said first portion and said second portion to generate a respective first intermediate result and a second intermediate result;
- summing said first intermediate result and said second intermediate result to generate a third intermediate result;
- applying a post-processing function on said third intermediate result to generate a fourth intermediate result;
- computing an output tensor by summing said received input tensor with said fourth intermediate result;
- recursing by setting said input tensor to said output tensor until said output tensor is of size one in a pre-determined dimension;
- and,
- performing a prediction of a speech sample using said output tensor of size one in a pre-determined dimension.
A system for generating speech samples comprising:
- a plurality of FFTNet blocks arranged in series, wherein each FFTNet block includes
- a splitter module,
- a convolution module,
- a summation block, and
- a post-processing module, wherein said post-processing module generates an output based upon said composite tensor,
- a fully connected layer, wherein said fully connected layer is coupled to a last FFTNet block in said series;
- and,
- a softmax classifier coupled to an output of said fully connected layer.
A computer program product including one or more non-transitory machine-readable mediums encoded with instructions that when executed by one or more processors cause a process to be carried out for generating speech samples, the process comprising:
- receiving an input tensor;
- splitting said received input tensor into a first portion and a second portion;
- performing a 1×1 convolution respectively on said first portion and said second portion to generate a respective first intermediate result and a second intermediate result;
- summing said first intermediate result and said second intermediate result to generate a third intermediate result;
- applying a post-processing function on said third intermediate result to generate a fourth intermediate result;
- computing an output tensor by summing said received input tensor with said fourth intermediate result;
- recursing by setting said input tensor to said output tensor until said output tensor is of size one in a pre-determined dimension;
- and,
- performing a prediction of a speech sample using said output tensor of size one in a pre-determined dimension.
Dependencies are inferred from the claim text.