Streaming speech synthesis method and system for supporting real-time conversation model

A real-time speech synthesis system comprising:

• one or more processors including:
• a conversation model configured to receive a streaming utterance of a user in real time and generate a response sentence incrementally in units of one or more texts in real time for responding to the streaming utterance of the user, wherein the conversation model incrementally outputs the units of one or more texts constituting the response sentence even when the response sentence is not fully completed;
• and
• a speech synthesis model configured to synthesize streaming speech waveforms corresponding to each of the units of the one or more texts as output by the conversation model, such that the synthesizing occurs before a complete response sentence corresponding to an entire utterance of the user is generated from the conversation model,
• wherein an input unit of the speech synthesis model matches an output unit of the conversation model, thereby minimizing a system delay through real-time interoperation between the conversation model and the speech synthesis model.

A computer-implemented method for real-time speech synthesis, the method comprising:

• synthesizing, by a machine-learning-based speech synthesis model implemented by a processor, streaming speech waveforms corresponding to partial text segments output incrementally by a machine-learning-based conversation model while the conversation model is still generating the response sentence,
• wherein the machine-learning-based conversation model incrementally outputs units of one or more texts constituting the response sentence even when the response sentence is not fully completed;
• and
• outputting, by a speech output module implemented by the processor, the synthesized speech in the unit of a text to a user device in real time concurrently while performing a process of the synthesizing,
• wherein an input unit of the machine-learning-based speech synthesis model matches an output unit of the machine-learning-based conversation model, thereby minimizing a system delay through real-time interoperation between the machine-learning-based conversation model and the machine-learning-based speech synthesis model.