System and method for generating expressive prosody for speech synthesis

A system for producing an expressive prosody model, comprising at least one hardware processor configured to:

• receive a plurality of non-neutral target prosody vector sequences describing a plurality of reference voice samples of one or more reference speakers, each prosody vector associated with one of a plurality of time instances;
• receive a plurality of reference textual features comprising a plurality of expression labels describing said plurality of reference voice samples, each label having a time instance selected from a plurality of non-neutral time instances selected from said plurality of time instances;
• produce a plurality of parallel neutral prosody vector sequences equivalent to said plurality of non-neutral target prosody vector sequences at said plurality of non-neutral time instances by applying a linear combination of a plurality of statistical measures computed using a plurality of sub-sequences of said plurality of target prosody vector sequences to said plurality of sub-sequences, where said plurality of sub-sequences is selected according to an identified proximity test applied to a plurality of neutral time instances identified in said plurality of time instances;
• and
• train at least one machine learning module using said plurality of non-neutral target prosody vector sequences and said plurality of parallel neutral prosody vector sequences to produce an expressive prosody model.

A system for producing speech, comprising at least one hardware processor configured to:

• access an expressive prosody model, wherein said expressive prosody model is generated by:
• receiving a plurality of non-neutral target prosody vector sequences describing a plurality of reference voice samples of one or more reference speakers, each prosody vector associated with one of a plurality of time instances;
• receiving a plurality of reference textual features comprising a plurality of expression labels describing said plurality of reference voice samples, each label having a time instance selected from a plurality of non-neutral time instances selected from said plurality of time instances;
• producing a plurality of parallel neutral prosody vector sequences equivalent to said plurality of non-neutral target prosody vector sequences at said plurality of non-neutral time instances by applying a linear combination of a plurality of statistical measures computed using a plurality of sub-sequences of said plurality of target prosody vector sequences to said plurality of sub-sequences, where said plurality of sub-sequences is selected according to an identified proximity test applied to a plurality of neutral time instances identified in said plurality of time instances;
• and
• training at least one machine learning module using said plurality of non-neutral target prosody vector sequences and said plurality of parallel neutral prosody vector sequences to produce an expressive prosody model;
• and
• using said expressive prosody model to produce an audio waveform from an input text.