Speech synthesis using one or more recurrent neural networks
AssigneeCerence Operating Company
InventorsVincent Pollet, Enrico Zovato
Life of this patent
Estimating the term…
Abstract
Aspects of the disclosure are related to synthesizing speech or other audio based on input data. Additionally, aspects of the disclosure are related to using one or more recurrent neural networks. For example, a computing device may receive text input; may determine features based on the text input; may provide the features as input to an recurrent neural network; may determine embedded data from one or more activations of a hidden layer of the recurrent neural network; may determine speech data based on a speech unit search that attempts to select, from a database, speech units based on the embedded data; and may generate speech output based on the speech data.
Claims20
A method comprising:
- receiving text input;
- determining unit-level features based on the text input;
- providing the features as input to a unit-level recurrent neural network (RNN);
- determining, by a computing device, unit-level embedded data from one or more activations of a hidden layer of the RNN, the RNN during training having an input layer, one or more hidden layers including said hidden layer, and an output layer;
- providing an input to a frame-level recurrent neural network based on an output of the unit-level recurrent neural network, the frame-level recurrent neural network producing successive frame-level outputs corresponding to respective acoustic frames of speech output;
- determining speech data based on a speech unit search, wherein the speech unit search selects, from a database, speech units using the unit-level embedded data as an input to the speech unit search,
- and wherein the speech data further depends on the successive frame-level outputs;
- and
- causing speech output to be generated based on the speech data.
An apparatus comprising:
- one or more processors;
- and
- memory storing executable instructions that, when executed by the one or more processors, cause the apparatus to:
- receive text input;
- determine unit-level features based on the text input;
- provide the features as input to a unit-level recurrent neural network (RNN);
- determine unit-level embedded data from one or more activations of a hidden layer of the RNN, the RNN during training having an input layer, one or more hidden layers including said hidden layer, and an output layer;
- provide an input to a frame-level recurrent neural network based on an output of the unit-level recurrent neural network, the frame-level recurrent neural network producing successive frame-level outputs corresponding to respective acoustic frames of speech output;
- determine speech data based on a speech unit search, wherein the speech unit search selects, from a database, speech units using the unit-level embedded data as an input to the speech unit search, and wherein the speech data further depends on the successive frame-level outputs;
- and
- cause speech output to be generated based on the speech data.
One or more non-transitory computer-readable storage media storing executable instructions that, when executed cause an apparatus to perform operations comprising:
- receiving text input;
- determining unit-level features based on the text input;
- providing the features as input to a unit-level recurrent neural network (RNN), the RNN during training having an input layer, one or more hidden layers including said hidden layer, and an output layer;
- providing an input to a frame-level recurrent neural network based on an output of the unit-level recurrent neural network, the frame-level recurrent neural network producing successive frame-level outputs corresponding to respective acoustic frames of speech output;
- determining embedded data from one or more activations of a hidden layer of the RNN;
- determining speech data based on a speech unit search, wherein the speech unit search selects, from a database, speech units using the embedded data as an input to the speech unit search, and wherein the speech data further depends on the successive frame-level outputs;
- and causing speech output to be generated based on the speech data.
Dependencies are inferred from the claim text.