# US 10,373,623 B2 · Apparatus and method for processing an audio signal to obtain a processed audio signal using a target time-domain envelope > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,373,623 B2 - Document ID: US10373623B2 - Status: Granted - Kind: B2 - Application number: 15/682,123 - Filed: 2017-08-21 - Published: 2017-11-30 - Granted: 2019-08-06 - Earliest claimed: 2015-02-26 - Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V. - Applicants: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V. - Inventors: Christian Dittmar; Meinard Mueller; Sascha Disch - Web page: https://patentgrep.com/patent/US10373623B2 ## CPC - G10L 19/03: Spectral prediction for preventing pre-echo; Temporary noise shaping [TNS], e.g. in MPEG2 or MPEG4 - G10L 13/04: Details of speech synthesis systems, e.g. synthesiser structure or memory management - G10L 21/0272: Voice signal separating - G10L 21/0388: Details of processing therefor - G10L 25/03: characterised by the type of extracted parameters - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Subject of the invention is an apparatus described by a schematic block diagram for processing an audio signal to obtain a processed audio signal. The apparatus includes a phase calculator for calculating phase values for spectral values of a sequence of frequency-domain frames representing overlapping frames of the audio signal. Moreover, the phase calculator is configured to calculate the phase values based on information on a target time-domain envelope related to the processed audio signal, so that the processed audio signal has at least in an approximation the target time-domain envelope and a spectral envelope determined by the sequence of frequency-domain frames. ## Claims ### Claim 1 An apparatus for processing an audio signal to acquire a processed audio signal, comprising: a phase calculator for calculating phase values for spectral values of a sequence of frequency-domain frames representing overlapping frames of the audio signal, wherein the phase calculator is configured to calculate the phase values based on information on a target time-domain envelope related to the processed audio signal, so that the processed audio signal comprises at least in an approximation the target time-domain envelope and a spectral envelope determined by the sequence of frequency-domain frames. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the phase calculator comprises: an iteration processor for performing an iterative algorithm to calculate, starting from initial phase values, the phase values for the spectral values using an optimization target entailing consistency of overlapping blocks in the overlapping range, wherein the iteration processor is configured to use, in a further iteration step, an updated phase estimate depending on the target time-domain envelope. ### Claim 3 (depends on 1) Apparatus of claim 1, wherein the phase calculator is configured to apply an amplitude modulation to an intermediate time domain reconstruction of an audio signal based on the target time domain envelope. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the phase calculator is configured to apply a convolution of a spectral representation of at least one target time-domain envelope and at least one intermediate frequency-domain reconstruction or selected parts or bands or only a high-pass portion or only several bandpass portions of the at least one target time-domain envelope or the at least one intermediate frequency-domain reconstruction of an audio signal. ### Claim 5 (depends on 3) The apparatus of claim 3, wherein the phase calculator comprises: a frequency-to-time converter for calculating the intermediate time-domain reconstruction of the audio signal from the sequence of frequency-domain frames and initial phase value estimates or phase value estimates of a preceding iteration step, an amplitude modulator for modulating the intermediate time-domain reconstruction using a target time-domain envelope to acquire an amplitude-modulated audio signal, and a time-to-frequency converter for converting the amplitude-modulated signal into a further sequence of frequency-domain frames comprising phase values, and wherein the phase calculator is configured to use, for a next iteration step, the phase values and the spectral values of the sequence of frequency-domain frames. ### Claim 6 (depends on 5) The apparatus of claim 5, wherein the phase calculator is configured to output the intermediate time-domain reconstruction as the processed audio signal, when an iteration determination condition is fulfilled. ### Claim 7 (depends on 4) The apparatus of claim 4, wherein the phase calculator comprises: a convolution processor for applying a convolution kernel and for applying a shift kernel and for adding an overlapping part of an adjacent frame of a central frame to the central frame to acquire the intermediate frequency-domain reconstruction of the audio signal. ### Claim 8 (depends on 4) The apparatus of claim 4, wherein the phase calculator is configured to use phase values acquired by the convolution as updated phase value estimates for a next iteration step. ### Claim 9 (depends on 4) The apparatus of claim 4, further comprising a target envelope converter for converting the target time-domain envelope into the spectral domain. ### Claim 10 (depends on 4) The apparatus of claim 4, further comprising: a frequency-to-time converter for calculating the time-domain reconstruction from the intermediate frequency-domain reconstruction using the phase value estimates acquired from a most recent iteration step and the sequence of frequency-domain frames. ### Claim 11 (depends on 4) The apparatus of claim 4, wherein the phase calculator comprises a convolution processor to process the sequence of frequency-domain frames, wherein the convolution processor is configured to apply a time-domain overlap-and-add procedure to the sequence of frequency-domain frames in the frequency-domain to determine the intermediate frequency-domain reconstruction. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein the convolution processor is configured to determine, based on a current frequency-domain frame, a portion of an adjacent frequency-domain frame which contributes to the current frequency-domain frame after time-domain overlap-and-add is performed in the frequency-domain, wherein the convolution processor is further configured to determine an overlapping position of the portion of the adjacent frequency-domain frame within the current frequency-domain frame and to perform an addition of the portions of adjacent frequency-domain frames with the current frequency-domain frame at the overlapping position. ### Claim 13 (depends on 11) The apparatus of claim 11, wherein the convolution processor is configured to frequency-to-time transform a time-domain synthesis and a time-domain analysis window to determine a portion of an adjacent frequency-domain frame which contributes to the current frequency-domain frame after time-domain overlap-and-add is performed in the frequency-domain, wherein the convolution processor is further configured to shift the position of the adjacent frequency-domain frame to an overlapping position within the current frequency-domain frame and to apply the portion of the adjacent frequency-domain frame to the current frame at the overlapping position. ### Claim 14 (depends on 1) The apparatus of claim 1, wherein the phase calculator is configured to perform the iterative algorithm in accordance with the iterative signal reconstruction procedure by Griffin and Lim. ### Claim 15 (depends on 1) An audio decoder, comprising: the apparatus of claim 1, and an input interface for receiving an encoded signal, the encoded signal comprising a representation of the sequence of frequency-domain frames and a representation of the target time-domain envelope. ### Claim 16 (depends on 1) An audio source separation processor, comprising: an apparatus for processing of claim 1, and a spectral masker for masking a spectrum of an original audio signal to acquire a modified audio signal input into the apparatus for processing, wherein the processed audio signal is a separated source signal related to the target time-domain envelope. ### Claim 17 (depends on 1) A bandwidth enhancement processor for processing an encoded audio signal, comprising: an enhancement processor for generating an enhancement signal from an audio signal band comprised by the encoded signal, and an apparatus for processing in accordance with claim 1, wherein the enhancement processor is configured to extract the target time-domain envelope from an encoded representation comprised by the encoded signal or from the audio signal band comprised by the encoded signal. ### Claim 18 A method for processing an audio signal to acquire a processed audio signal, comprising: calculating phase values for spectral values of a sequence of frequency-domain frames representing overlapping frames of the audio signal, wherein the phase values are calculated based on information on a target time-domain envelope related to the processed audio signal, so that the processed audio signal comprises at least in an approximation the target time-domain envelope and a spectral envelope determined by the sequence of frequency-domain frames. ### Claim 19 A method of audio decoding, comprising: the method for processing an audio signal to acquire a processed audio signal, comprising: calculating phase values for spectral values of a sequence of frequency-domain frames representing overlapping frames of the audio signal, wherein the phase values are calculated based on information on a target time-domain envelope related to the processed audio signal, so that the processed audio signal comprises at least in an approximation the target time-domain envelope and a spectral envelope determined by the sequence of frequency-domain frames; receiving an encoded signal, the encoded signal comprising a representation of the sequence of frequency-domain frames, and a representation of the target time-domain envelope. ### Claim 20 A method of audio source separation, comprising: the method for processing an audio signal to acquire a processed audio signal, comprising: calculating phase values for spectral values of a sequence of frequency-domain frames representing overlapping frames of the audio signal, wherein the phase values are calculated based on information on a target time-domain envelope related to the processed audio signal, so that the processed audio signal comprises at least in an approximation the target time-domain envelope and a spectral envelope determined by the sequence of frequency-domain frames, and masking a spectrum of an original audio signal to acquire a modified audio signal input into the apparatus for processing; wherein the processed audio signal is a separated source signal related to the target time-domain envelope. ### Claim 21 A method of bandwidth enhancement of an encoded audio signal, comprising: generating an enhancement signal from an audio signal band comprised by the encoded signal; the method for processing an audio signal to acquire a processed audio signal, comprising: calculating phase values for spectral values of a sequence of frequency-domain frames representing overlapping frames of the audio signal, wherein the phase values are calculated based on information on a target time-domain envelope related to the processed audio signal, so that the processed audio signal comprises at least in an approximation the target time-domain envelope and a spectral envelope determined by the sequence of frequency-domain frames; wherein the generating comprises extracting the target time-domain envelope from an encoded representation comprised by the encoded signal or from the audio signal band comprised by the encoded signal. ### Claim 22 A non-transitory digital storage medium having a computer program stored thereon to perform a method for processing an audio signal to acquire a processed audio signal, the method comprising: calculating phase values for spectral values of a sequence of frequency-domain frames representing overlapping frames of the audio signal, wherein the phase values are calculated based on information on a target time-domain envelope related to the processed audio signal, so that the processed audio signal comprises at least in an approximation the target time-domain envelope and a spectral envelope determined by the sequence of frequency-domain frames, when said computer program is run by a computer. ### Claim 23 A non-transitory digital storage medium having a computer program stored thereon to perform a method of audio decoding, the method comprising: the method for processing an audio signal to acquire a processed audio signal, comprising: calculating phase values for spectral values of a sequence of frequency-domain frames representing overlapping frames of the audio signal, wherein the phase values are calculated based on information on a target time-domain envelope related to the processed audio signal, so that the processed audio signal comprises at least in an approximation the target time-domain envelope and a spectral envelope determined by the sequence of frequency-domain frames; receiving an encoded signal, the encoded signal comprising a representation of the sequence of frequency-domain frames, and a representation of the target time-domain envelope, when said computer program is run by a computer. ### Claim 24 A non-transitory digital storage medium having a computer program stored thereon to perform a method of audio source separation, the method comprising: the method for processing an audio signal to acquire a processed audio signal, comprising: calculating phase values for spectral values of a sequence of frequency-domain frames representing overlapping frames of the audio signal, wherein the phase values are calculated based on information on a target time-domain envelope related to the processed audio signal, so that the processed audio signal comprises at least in an approximation the target time-domain envelope and a spectral envelope determined by the sequence of frequency-domain frames, and masking a spectrum of an original audio signal to acquire a modified audio signal input into the apparatus for processing; wherein the processed audio signal is a separated source signal related to the target time-domain envelope, when said computer program is run by a computer. ### Claim 25 A non-transitory digital storage medium having a computer program stored thereon to perform a method of bandwidth enhancement of an encoded audio signal, the method comprising: generating an enhancement signal from an audio signal band comprised by the encoded signal; the method for processing an audio signal to acquire a processed audio signal, comprising: calculating phase values for spectral values of a sequence of frequency-domain frames representing overlapping frames of the audio signal, wherein the phase values are calculated based on information on a target time-domain envelope related to the processed audio signal, so that the processed audio signal comprises at least in an approximation the target time-domain envelope and a spectral envelope determined by the sequence of frequency-domain frames; wherein the generating comprises extracting the target time-domain envelope from an encoded representation comprised by the encoded signal or from the audio signal band comprised by the encoded signal, when said computer program is run by a computer.