Method and system for a parametric speech synthesis

A speech conversion method implemented in a computer system comprising a processor, memory accessible by the processor, and computer program instructions stored in the memory and executable by the processor, the method comprising:

• storing a plurality of phonemes in the memory;
• converting input text to at least one phoneme selected from the plurality of phonemes stored in the memory;
• detecting a language of the input text;
• storing, in a machine-learning model at the computer system, voice patterns for a plurality of individuals, receiving, at the machine-learning model at the computer system, the at least one phoneme and an identity of a speaker, and generating and enhancing, with the machine-learning model at the computer system, acoustic features for each of the at least one phoneme, wherein the machine-learning model comprises a deep learning neural network that uses (i) phoneme embedding, (ii) speaker embedding, and (iii) language embedding, and wherein the voice patterns comprise a plurality of production components and a plurality of acoustic components arranged in a matrix, each of the plurality of production components contributing a selected amount to each of the acoustic components, such that the selected amount contributed is present in the matrix for each pair of production component and acoustic component, and wherein enhancing acoustic features comprises at least one of spectral enhancement or focal enhancement;
• and
• receiving, at the computer system, the generated acoustic features from the machine learning model, and generating, at the computer system, a speech signal simulating a voice of the identified speaker in the language, wherein the generated acoustic features include accent acoustic features such that the generated speech signal further simulates the voice of the identified speaker in the language and in an accent.

A speech conversion system comprising;

• a memory for storing a plurality of phonemes;
• a text converter adapted to convert input text to at least one phoneme selected from the plurality of phonemes stored in the memory, wherein the text converter is further adapted to detect a language of the input text;
• a machine-learning model storing voice patterns for a plurality of individuals and adapted to receive the at least one phoneme from the text converter and an identity of a speaker and to generate and enhance acoustic features for each of the at least one phoneme, wherein the machine-learning model comprises a deep learning neural network that uses (i) phoneme embedding, (ii) speaker embedding, and (iii) language embedding,
• wherein the voice patterns comprise a plurality of production components and a plurality of acoustic components arranged in a matrix, each of the plurality of production components contributing a selected amount to each of the acoustic components, such that the selected amount contributed is present in the matrix for each pair of production component and acoustic component, and having voice pattern options, and wherein the enhanced acoustic features comprise at least one of spectral enhancement or focal enhancement;
• and
• a decoder adapted to receive the generated acoustic features from the machine-learning model and to generate a speech signal simulating a voice of the identified speaker in the language, wherein the generated acoustic features include accent acoustic features such that the generated speech signal further simulates the voice of the identified speaker in the language and in an accent.