Sound synthesis method, sound synthesis apparatus, and recording medium storing instructions to perform sound synthesis method

A sound synthesis method performed by a sound synthesis apparatus including a transceiver and a processor, comprising:

• obtaining a plurality of sound samples;
• preprocessing each sound sample to convert each sound sample into a spectrogram;
• determining whether each sound sample corresponds to a single-frame or multi-frame input and, based thereon, selectively inputting the spectrogram to either a fully connected encoder layer if each sound sample corresponds to the single-frame or a convolutional encoder layer if each sound sample corresponds to the multi-frame;
• generating a plurality of latent codes by inputting the spectrogram of each sound sample to an encoder of an artificial neural network pre-trained to output a latent code that maximizes timbre information;
• generating one synthesized latent code by synthesizing the plurality of latent codes based on a weight present for each sound sample;
• and
• generating a synthesized sound by inputting the synthesized latent code to a decoder of the pre-trained artificial neural network.

A computer-readable recording medium storing a computer program including commands for causing a processor to perform a method comprising:

• obtaining a plurality of sound samples;
• preprocessing each sound sample to convert each sound sample into a spectrogram;
• determining whether each sound sample corresponds to a single-frame or multi-frame input and, based thereon, selectively inputting the spectrogram to either a fully connected encoder layer if each sound sample corresponds to the single-frame or a convolutional encoder layer if each sound sample corresponds to the multi-frame;
• generating a plurality of latent codes by inputting the spectrogram of each sound sample to an encoder of an artificial neural network pre-trained to output a latent code that maximizes timbre information;
• generating one synthesized latent code by synthesizing the plurality of latent codes based on a weight present for each sound sample;
• and
• generating a synthesized sound by inputting the synthesized latent code to a decoder of the pre-trained artificial neural network.