Audio signal processing apparatus and method for deep neural network-based audio encoder and decoder

An audio signal processing apparatus comprising:

• a processor electronically communicating with a deep neural network,
• wherein the processor is configured to:
• acquire an input signal before encoding and an output signal after quantization and decoding;
• calculate a perceptual global loss for a frame corresponding to the input and the output signals;
• acquire a plurality of subframes corresponding to the input and output signals by applying a windowing function to the frame of the input and output signals;
• calculate perceptual local losses for the plurality of subframes corresponding to the input and output signals;
• acquire multi-time scale perceptual loss based on the perceptual global and local losses, and
• control an encoder based on the deep neural network to encode the input signal,
• wherein the processor is further configured to:
• control the encoder to pass the input signal through a first gated linear unit (GLU) block within the encoder;
• downsample the signal passed through the first GLU block;
• control the encoder to pass the downsampled signal through a second GLU block;
• and
• control the encoder to generate a latent vector through a first long-short term memory (LSTM) block based on the signal passed through the second GLU block.

An audio signal processing method executed by a processor in a computing system including an encoder, a decoder, and a deep neural network, the method comprising:

• acquiring, by the processor, an input signal before encoding and an output signal after quantization and decoding;
• calculating, by the processor, a perceptual global loss for a frame corresponding to the input and the output signals;
• acquiring, by the processor, a plurality of subframes corresponding to the input and output signals by applying a windowing function to the frame of the input and output signals;
• calculating, by the processor, perceptual local losses for the plurality of subframes corresponding to the input and output signals;
• acquiring, by the processor, a multi-time scale perceptual loss based on the perceptual global and local losses;
• controlling, by the processor, an encoder to pass the input signal through a first gated linear unit (GLU) block within the encoder;
• downsampling, by the processor, the signal passed through the first GLU block;
• controlling, by the processor, the encoder to pass the downsampled signal through a second GLU block;
• and
• controlling, by the processor, the encoder to generate a latent vector through a first long-short-term memory (LSTM) block based on the signal passed through the second GLU block.