Transformer-based voice recognition technology using improved voice as conditioning feature

A voice recognition method performed by a voice recognition system, comprising:

• providing voice data comprising noise;
• estimating clean voice data from the voice data using a voice improvement model trained to remove the noise from the voice data to estimate and generate the clean voice data;
• providing a voice recognition model trained to perform voice recognition robust against noise based on a voice feature of the voice data comprising the noise and a voice feature of the estimated clean voice data without the noise, the voice recognition model comprising a feature-wise layer modulation (FILM) layer, wherein the FiLM layer is added to an encoder of an end-to-end voice recognition model of a transformer structure comprising a self-attention layer;
• inputting the estimated clean voice data without the noise to a modulator of the voice recognition model;
• generating a scale value and a bias value by the modulator by applying an affine transform to the voice feature of the estimated clean voice data without the noise to adaptively select required information from the estimated clean voice data for reducing an influence from the noise;
• inputting the scale value and the bias value to the FILM layer, and
• inputting the voice data comprising the noise to the FILM layer, the estimated clean voice data without the noise being a conditioning feature of the voice recognition model and the voice data comprising the noise being an input feature of the voice recognition model;
• and
• performing voice recognition based on the estimated clean voice data and the voice data comprising the noise using the voice recognition model by performing an element-wise product of the scale value and the voice feature of the voice data comprising the noise and adding the bias value thereto in the FILM layer.