Audio signal generation model and training method using generative adversarial network

A learning method of a generative adversarial network-based audio signal generation model executed by a processor, wherein the method comprising:

• (a) generating, by a generator, an audio signal;
• (b) separating the generated audio signal into a harmonic component signal and a percussive component signal using a harmonic-percussive separation model;
• (c) evaluating, by a first discriminator, whether the harmonic component signal is real or fake, and
• (d) evaluating, by a second discriminator, whether the percussive component signal is real or fake,
• wherein the first discriminator has a first kernel dilation factor greater than a second kernel dilation factor of the second discriminator, and the first discriminator has a first receptive field greater than a second receptive field of the second discriminator,
• wherein the generator is trained to minimize errors between samples of real signals and audio signals generated by the generator, using a restoration loss function applied to the generator, in a first phase training, and
• wherein (a) to (d) are performed repeatedly for the generator, the harmonic-percussive separation model, the first discriminator, and the second discriminator to learn in a backward propagation manner for adversarial training through end-to-end learning after the first phase training, as a second phase training.

An apparatus for generating an audio signal using a generative adversarial network, the apparatus comprising:

• a memory configured to store at least one instruction;
• a processor configured to execute the at least one instruction stored in the memory,
• a generator generating an audio signal with an external input;
• a harmonic-percussive separation model separating the generated audio signal into a harmonic component signal and a percussive component signal;
• a first discriminator evaluating whether the harmonic component signal is real or fake;
• and
• a second discriminator evaluating whether the percussive component signal is real or fake,
• wherein the first discriminator has a first kernel dilation factor greater than a second kernel dilation factor of the second discriminator, and the first discriminator has a first receptive field greater than a second receptive field of the second discriminator,
• wherein the processor is configured to:
• train the generator to minimize errors between samples of real signals and audio signals generated by the generator, using a restoration loss function applied to the generator, in a first phase training, and
• adversarial train the generator, the harmonic-percussive separation model, the first discriminator, and the second discriminator, through end-to-end learning, after the first phase training, in a second phase training.