End-to-end multi-task denoising for joint signal distortion ratio (SDR) and perceptual evaluation of speech quality (PESQ) optimization
AssigneeSamsung Electronics Co., Ltd.
InventorsJaeyoung Kim, Mostafa El-Khamy, Jungwon Lee
Life of this patent
Estimating the term…
Abstract
A method and system for providing end-to-end multi-task denoising for joint signal distortion ratio (SDR) and perceptual evaluation of speech quality (PESQ) optimization is herein disclosed. According to one embodiment, an method includes receiving a noisy signal, generating a denoised output signal, determining a signal distortion ratio (SDR) loss function based on the denoised output signal, determining a perceptual evaluation of speech quality (PESQ) loss function based on the denoised output signal, and optimizing an overall loss function based on the PESQ loss function and the SDR loss function.
Claims19
A method for training a neural network, comprising:
- receiving a noisy signal and performing short-term Fourier transform (STFT) on the noisy signal;
- generating a denoised output signal from the noisy signal and performing inverse short-term Fourier transform (ISTFT) on the denoised output signal;
- determining a signal distortion ratio (SDR) loss function based on the denoised output signal after the ISTFT, wherein the SDR loss function uses a scale invariant (SI) SDR metric minimizing spectra mismatch;
- determining a perceptual evaluation of speech quality (PESQ) loss function based on the denoised output signal after the ISTFT, wherein the PESQ loss function approximates symmetric and asymmetric disturbance of PESQ minimizing metric mismatch;
- and
- optimizing an overall loss function based on the PESQ loss function and the SDR loss function.
A system for training a neural network, comprising:
- a memory;
- and
- a processor configured to:
- receive a noisy signal and perform short-term Fourier transform (STFT) on the noisy signal;
- generate a denoised output signal from the noisy signal and perform inverse short-term Fourier transform (ISTFT) on the denoised output signal;
- determine a signal distortion ratio (SDR) loss function based on the denoised output signal after the ISTFT, wherein the SDR loss function uses a scale invariant (SI) SDR metric minimizing spectra mismatch;
- determine a perceptual evaluation of speech quality (PESQ) loss function based on the denoised output signal after the ISTFT, wherein the PESQ loss function approximates symmetric and asymmetric disturbance of a PESQ minimizing metric mismatch;
- and
- optimize an overall loss function based on the PESQ loss function and the SDR loss function.
A method of training a neural network, comprising:
- receiving a time-aligned pair of a noisy signal and a denoised signal;
- performing short-term Fourier transform (STFT) on the noisy signal and the denoised signal;
- generating a denoised output signal from the noisy signal and performing inverse short-term Fourier transform (ISTFT) on the denoised output signal;
- determining a signal distortion ratio (SDR) loss function based on the denoised output signal after the ISTFT, wherein the SDR loss function uses a scale invariant (SI) SDR metric minimizing spectra mismatch;
- performing disturbance processing on the noisy signal and the denoised signal;
- determining a perceptual evaluation of speech quality (PESQ) loss function based on an aggregation of the disturbance processing, wherein the PESQ loss function approximates symmetric and asymmetric disturbance of PESQ minimizing metric mismatch;
- and
- optimizing an overall loss function based on the PESQ loss function and the SDR loss function.
Dependencies are inferred from the claim text.