# US 12,763,073 B2 · Real-time dynamic noise reduction using convolutional networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,763,073 B2 - Document ID: US12763073B2 - Status: Granted - Kind: B2 - Application number: 18/770,496 - Filed: 2024-07-11 - Published: 2025-02-06 - Granted: 2026-10-13 - Earliest claimed: 2020-09-25 - Assignees: Intel Corporation - Applicants: Intel Corporation - Inventors: Adam Kupryjanow; Tomasz Noczynski; Lukasz Pindor; Sebastian Rosenkiewicz - Web page: https://patentgrep.com/patent/US12763073B2 ## CPC - G10L 15/20: Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech (G10L21/02 takes precedence) - G10L 15/16: using artificial neural networks - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A system, method and computer readable medium for dynamic noise reduction in a voice call. The system includes an encoder having a short-time Fourier transform module to determine a magnitude spectrum and a phase spectrum of an input audio signal. The input audio signal includes speech and dynamic noise. A separator is coupled to the encoder. The separator comprises a temporal convolution network (TCN) used to develop a separation mask using the magnitude spectrum as input. The TCN is trained using a frequency SNR function used to calculate loss during training. A mixer is coupled to the separator to multiply the separation mask with the magnitude spectrum to separate the speech from the dynamic noise to obtain a denoise magnitude spectrum. The system also includes a decoder coupled to the mixer and the encoder. The decoder includes an inverse short-time Fourier transform module to reconstruct the input audio signal without the dynamic noise using the denoise magnitude spectrum and the phase spectrum. ## Claims ### Claim 1 A sound separation system, comprising: an encoder comprising a short-time Fourier transform module to determine a first magnitude spectrum and a phase spectrum of an input audio signal, the input audio signal including voice; a separator, coupled to the encoder, comprising a temporal convolution network (TCN) used to develop a separation mask using the first magnitude spectrum as input, wherein the TCN includes non-causal convolution layers and causal convolution layers to form a hybrid TCN architecture; and a mixer, coupled to the separator, to multiply the separation mask with the magnitude spectrum to separate the voice from the input audio signal to obtain a second magnitude spectrum for the voice. ### Claim 2 (depends on 1) The system of claim 1, wherein the audio signal includes additional sound and wherein the mixer is configured to separate the voice from the additional sound. ### Claim 3 (depends on 2) The system of claim 2, further comprising a decoder, coupled to the mixer and the encoder, comprising an inverse short-time Fourier transform module to reconstruct the input audio signal without the additional sound using the second magnitude spectrum and the phase spectrum. ### Claim 4 (depends on 1) The system of claim 1, wherein the non-causal convolution layers provide lookahead at future frames. ### Claim 5 (depends on 4) The system of claim 4, wherein the non-causal convolution layers increase latency of the system. ### Claim 6 (depends on 1) The system of claim 1, wherein the TCN comprises at least one stack of 1-D convolution blocks that repeat n times. ### Claim 7 (depends on 1) The system of claim 1, wherein the TCN is trained using a frequency SNR cost function used to calculate loss during training, and wherein the frequency SNR cost function includes a logarithmic scale to balance quiet and loud magnitudes. ### Claim 8 (depends on 1) The system of claim 1, wherein the system is executable on small form factor devices. ### Claim 9 A method for dynamic noise reduction, comprising: receiving, by an encoder, an input audio signal, the input audio signal including voice; performing, by the encoder, a short-time Fourier transform on the audio signal to generate a first magnitude spectrum and a phase spectrum; estimating, by a temporal convolution network (TCN), a separation mask based on the first magnitude spectrum using deep learning, wherein the TCN comprises non-causal convolution layers merged with causal convolution layers; and mixing the separation mask with the first magnitude spectrum to separate the voice from the input audio signal and obtain a second magnitude spectrum for the voice. ### Claim 10 (depends on 9) The method of claim 9, wherein the audio signal includes additional sound and wherein mixing includes separating the voice from the additional sound. ### Claim 11 (depends on 10) The method of claim 10, further comprising performing, by a decoder, an inverse short-time Fourier transform using the second magnitude spectrum and the phase spectrum to reconstruct the input audio signal without the additional sound. ### Claim 12 (depends on 9) The method of claim 9, wherein estimating the separation mask includes looking ahead at future frames. ### Claim 13 (depends on 9) The method of claim 9, wherein the TCN comprises at least one stack of 1-D dilated convolution blocks that repeat n times to estimate the separation mask using deep learning. ### Claim 14 (depends on 9) The method of claim 9, wherein the TCN is trained using a frequency SNR cost function used to calculate loss during training, and, wherein the frequency SNR cost function includes a logarithmic scale to balance quiet and loud magnitudes. ### Claim 15 At least one non-transitory computer readable medium, comprising a set of instructions, which when executed by one or more computing devices, cause the one or more computing devices to: receive, by an encoder, an input audio signal, the input audio signal including voice; perform, by the encoder, a short-time Fourier transform on the audio signal to generate a first magnitude spectrum and a phase spectrum; estimate, by a temporal convolution network (TCN), a separation mask based on the first magnitude spectrum using deep learning, wherein the TCN comprises non-causal convolution layers merged with causal convolution layers; and mix the separation mask with the first magnitude spectrum to generate a second magnitude spectrum for the voice. ### Claim 16 (depends on 15) The at least one non-transitory computer readable medium of claim 15, wherein the audio signal includes additional sound and wherein the set of instructions further cause the one or more computing devices to separate the voice from the additional sound. ### Claim 17 (depends on 16) The at least one non-transitory computer readable medium of claim 16, wherein the set of instructions further cause the one or more computing devices to: perform, by a decoder, an inverse short-time Fourier transform using the second magnitude spectrum and the phase spectrum to reconstruct the input audio signal without the additional sound. ### Claim 18 (depends on 15) The at least one non-transitory computer readable medium of claim 15, wherein estimating the separation mask includes looking ahead at future frames. ### Claim 19 (depends on 15) The at least one non-transitory computer readable medium of claim 15, wherein the TCN comprises at least one stack of 1-D dilated convolution blocks that repeat n times to estimate the separation mask using deep learning. ### Claim 20 (depends on 15) The at least one non-transitory computer readable medium of claim 15, wherein the TCN is trained using a frequency SNR cost function used to calculate loss during training, and, wherein the frequency SNR cost function includes a logarithmic scale to balance quiet and loud magnitudes.