# US 2023/0274754 A1 · Method for enhancing quality of audio data, and device using the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274754 A1
- Document ID: US20230274754A1
- Status: Published application
- Kind: A1
- Application number: 18/031,268
- Filed: 2020-11-20
- Published: 2023-08-31
- Earliest claimed: 2020-10-19
- Assignees: Deephearing Inc.; The Industry & Academic Cooperation in Chungnam National University (IAC)
- Applicants: Deephearing Inc.; The Industry & Academic Cooperation in Chungnam National University (IAC)
- Inventors: Kanghun AHN; Sungwon KIM
- Web page: https://patentgrep.com/patent/US20230274754A1

## CPC

- G10L 21/0232: Processing in the frequency domain
- G10L 21/0264: characterised by the type of parameter measurement, e.g. correlation techniques, zero crossing techniques or predictive techniques
- G10L 25/18: the extracted parameters being spectral information of each sub-band
- G10L 25/30: using neural networks
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Provided is a method of enhancing quality of audio data which comprise obtaining a spectrum of mixed audio data including noise, inputting two-dimensional (2D) input data corresponding to the spectrum to a convolutional network including a downsampling process and an upsampling process to obtain output data of the convolutional network, generating a mask for removing noise included in the audio data based on the obtained output data and removing noise from the mixed audio data using the generated mask, wherein, in the convolutional network, the downsampling process and the upsampling process are performed on a first axis of the 2D input data, and remaining processes other than the downsampling process and the upsampling process are performed on the first axis and a second axis.

## Claims

### Claim 1

A method of enhancing quality of audio data, the method comprising:
obtaining a spectrum of mixed audio data including noise;
inputting two-dimensional (2D) input data corresponding to the spectrum to a convolutional network including a downsampling process and an upsampling process to obtain output data of the convolutional network;
generating a mask for removing noise included in the audio data based on the obtained output data; and
removing noise from the mixed audio data using the generated mask,
wherein, in the convolutional network which is a U-NET convolutional network, the downsampling process and the upsampling process are performed only on a frequency axis of the 2D input data, and remaining processes other than the downsampling process and the upsampling process are performed on the frequency axis and a time axis, and
wherein the method further comprises:
performing a causal convolution on the 2D input data on the time axis,
wherein the performing of the causal convolution comprises:
performing zero padding on data of a preset size corresponding to the past relative to the time axis in the 2D input data.

### Claim 2

(canceled)

### Claim 3

(canceled)

### Claim 4

(canceled)

### Claim 5 (depends on 1)

The method of claim 1, wherein the performing of the causal convolution is performed on the time axis.

### Claim 6 (depends on 1)

The method of claim 1, wherein a batch normalization process is performed before the downsampling process.

### Claim 7 (depends on 1)

The method of claim 1, wherein the obtaining of the spectrum of mixed audio data including noise comprises:
obtaining the spectrum by applying a short-time Fourier transform (STFT) to the mixed audio data including noise.

### Claim 8 (depends on 1)

The method of claim 1, the method being performed on the audio data collected in real time.

### Claim 9

An audio data processing device comprising:
an audio data pre-processor configured to obtain a spectrum of mixed audio data including noise;
an encoder and a decoder configured to input 2D input data corresponding to the spectrum to a convolutional network including a downsampling process and an upsampling process to obtain output data of the convolutional network; and
an audio data post-processor configured to generate a mask for removing noise included in the audio data based on the obtained output data, and to remove noise from the mixed audio data using the generated mask,
wherein, in the convolutional network which is a U-NET convolutional network, the downsampling process and the upsampling process are performed only on a frequency axis of the 2D input data, and remaining processes other than the downsampling process and the upsampling process are performed on the frequency axis and a time axis, and
wherein the encoder and the decoder performs a causal convolution on the 2D input data on the time axis, and
wherein the causal convolution performs zero padding on data of a preset size corresponding to the past relative to the time axis in the 2D input data.