Text-to-speech (TTS) processing
AssigneeAmazon Technologies, Inc.
InventorsRoberto Barra Chicote, Adam Franciszek Nadolski, Thomas Edward Merritt, Bartosz Putrycz, Andrew Paul Breen
Life of this patent
Estimating the term…
Abstract
A speech model includes a sub-model corresponding to a vocal attribute. The speech model generates an output waveform using a sample model, which receives text data, and a conditioning model, which receives text metadata and produces a prosody output for use by the sample model. If, during training or runtime, a different vocal attribute is desired or needed, the sub-model is re-trained or switched to a different sub-model corresponding to the different vocal attribute.
Claims20
A computer-implemented method for generating audio data corresponding to different vocal attributes, the method comprising:
- generating, using a speech model and input text data, first audio output data corresponding to a first vocal attribute, wherein generating the first audio output data using the speech model comprises:
- generating, using a conditioning model, conditioning data using input text metadata, the conditioning data corresponding to at least one of pitch, rate, and volume,
- generating, using a sample model, audio sample data corresponding to the input text data and conditioning data, and
- generating, using an output model and a first sub-model corresponding to the first vocal attribute, audio output data using the audio sample data, the audio output data corresponding to a response to a query corresponding to the input text data, wherein the first vocal attribute includes at least one of a style, accent, tone, and language;
- and
- receiving a request to change from the first vocal attribute to a second vocal attribute;
- determining that a second sub-model corresponds to the second vocal attribute;
- selecting a second speech model including the sample model, the conditioning model, the output model, and the second sub-model;
- and
- generating, using the second speech model, second audio output data corresponding to the second vocal attribute.
A computer-implemented method comprising:
- receiving text data;
- receiving text metadata corresponding to the text data;
- generating, using the text metadata and a conditioning model, conditioning data;
- generating, using the text data, the conditioning data, a first sub-model of a speech model, and the speech model, first audio output data corresponding to a first vocal attribute;
- receiving a request to change from the first vocal attribute to a second vocal attribute;
- determining that a second sub-model of the speech model corresponds to the second vocal attribute;
- and
- generating, using second text data, second conditioning data, the second sub-model, and the speech model, second audio output data corresponding to the second vocal attribute.
A system comprising:
- at least one processor;
- and
- at least one memory including instructions that, when executed by the at least one processor, cause the system to:
- receive text data;
- receive text metadata corresponding to the text data;
- generate, using the text metadata and a conditioning model, conditioning data;
- generate, using the text data, the conditioning data, a first sub-model of a speech model, and the speech model, first audio output data corresponding to a first vocal attribute;
- receive a request to change from the first vocal attribute to a second vocal attribute
- determine that a second sub-model of the speech model corresponds to the second vocal attribute;
- and
- generate, using second text data, second conditioning data, the second sub-model, and the speech model, second audio output data corresponding to the second vocal attribute.
Dependencies are inferred from the claim text.