Speech synthesizer using artificial intelligence, method of operating speech synthesizer and computer-readable recording medium

A method of operating a speech synthesizer, the method comprising:

• storing a plurality of sentences and prior information of a word classified into a minor class among a plurality of classes with respect to each sentence, wherein the plurality of classes comprises a first class corresponding to a first reading break with a first time interval, a second class corresponding to a second reading break with a second time interval greater than the first time interval, and a third class corresponding to a third reading break with a greater time interval than the second time interval, wherein the minor class has a smallest count of phrases from a same type of word phrase class among the first to third classes in one sentence;
• and
• determining an oversampling rate of the word based on the prior information;
• determining a number of times of oversampling of the word using the determined oversampling rate;
• generating sentences including the word labeled with a reading break of the word based on the determined number of times of oversampling;
• training a synthesized speech model for predicting the reading break of the word using a training set including the word and a sentence labeled with the reading break of the word;
• based on a new sentence being input to the synthesized speech model, outputting a first probability that each word in the new sentence belongs to the first class, a second probability that each word in the new sentence belongs to the second class, and a third probability that each word in the new sentence belongs to the third class;
• determining a largest value of the first to third probabilities as a class indicating the reading break of each word;
• and
• output synthesized speech based on at least the indicated reading break of each word.

A non-transitory computer-readable recording medium for performing a method of operating a speech synthesizer, the method comprising:

• storing a plurality of sentences and prior information of a word classified into a minor class among a plurality of classes with respect to each sentence, wherein the plurality of classes comprises a first class corresponding to a first reading break with a first time interval, a second class corresponding to a second reading break with a second time interval greater than the first time interval, and a third class corresponding to a third reading break with a greater time interval than the second time interval, wherein the minor class has a smallest count of phrases from a same type of word phrase class among the first to third classes in one sentence;
• and
• determining an oversampling rate of the word based on the prior information of the word;
• determining a number of times of oversampling of the word using the determined oversampling rate;
• generating sentences including the word labeled with a reading break of the word based on the determined number of times of oversampling;
• train a synthesized speech model for predicting the reading break of the word using a training set including a sentence including the word and a sentence labeled with the reading break of the word,
• based on a new sentence being input to the synthesized speech model, output a first probability that each word in the new sentence belongs to the first class, a second probability that each word in the new sentence belongs the second class, and a third probability that each word configuring the new sentence belongs to the third class,
• determine a largest value of the first to third probabilities as a class indicating the reading break of each word, and
• output synthesized speech based on at least the indicated reading break of each word.