Robustness/performance improvement for deep learning based speech enhancement against artifacts and distortion
AssigneeDolby Laboratories Licensing Corporation
InventorsJundai Sun, Lie Lu, Zhiwei Shuang
Life of this patent
Estimating the term…
Abstract
Described is a method of processing an audio signal. The method includes a first step for applying enhancement to a first component of the audio signal and/or applying suppression to a second component of the audio signal relative to the first component, and a second step of modifying an output of the first step by applying a deep learning based model to the output of the first step, for perceptually improving the first component of the audio signal. Also described is an apparatus for carrying out the method, as well as corresponding programs and computer-readable storage media.
Claims17
A method of processing an audio signal, comprising:
- a first step for applying enhancement to a first component of the audio signal and/or applying suppression to a second component of the audio signal relative to the first component, wherein an output of the first step is an initial transform domain mask indicating weighting coefficients for individual bins or bands, and wherein applying the initial transform domain mask to the audio signal results in the enhancement of the first component and/or the suppression of the second component relative to the first component;
- and
- a second step of modifying the output of the first step by applying a deep learning based model to the output of the first step, for perceptually improving the first component of the audio signal by removing artifacts and/or distortions introduced in the audio signal by the first step, wherein the deep learning model generates an improved mask relative to the initial transform domain mask by taking, as input, the initial transform domain mask and generating the improved mask that removes mask noise present in the initial transform domain mask.
An apparatus for processing an audio signal, comprising:
- a first stage for applying enhancement to a first component of the audio signal and/or applying suppression to a second component of the audio signal relative to the first component, wherein an output of the first stage is an initial transform domain mask indicating weighting coefficients for individual bins or bands, and wherein applying the initial transform domain mask to the audio signal results in the enhancement of the first component and/or the suppression of the second component relative to the first component;
- and
- a second stage for modifying the output of the first stage by applying a deep learning based model to the output of the first stage, for perceptually improving the first component of the audio signal by removing artifacts and/or distortions introduced in the audio signal by the first stage, wherein the deep learning model generates an improved mask relative to the initial transform domain mask by taking, as input, the initial transform domain mask and generating the improved mask that removes mask noise present in the initial transform domain mask.
Dependencies are inferred from the claim text.