Method and computer readable storage medium for automated speech recognition using retrieval-based voice conversion

A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by a computer, cause the computer to perform a method for generating speech from a source voice of a speaker in a language for a speech device connected to the computer, where the speech device language is a non-native language of the speaker, the method comprising:

• receiving, via the speech device, speech in the source voice;
• encoding, by a phoneme encoder, the speech in the source voice into a content vector;
• retrieving top k similar content vectors from a vector representation database that match phonetic patterns of the encoded source voice;
• generating, using a Mel-Spectrogram Generative Model, a new voice waveform based on the similar content vectors that contains linguistic content of the source voice, but with the phonetic patterns of a native speaker's voice of the speech device language;
• and
• outputting the new voice waveform.