# US 10,484,808 B2 · Audio signal processing apparatus and method for processing an input audio signal > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,484,808 B2 - Document ID: US10484808B2 - Status: Granted - Kind: B2 - Application number: 15/920,006 - Filed: 2018-03-13 - Published: 2018-07-26 - Granted: 2019-11-19 - Earliest claimed: 2016-03-16 - Assignees: Huawei Technologies Co., Ltd. - Applicants: Huawei Technologies Co., Ltd. - Inventors: Juergen Geiger; Peter Grosche - Web page: https://patentgrep.com/patent/US10484808B2 ## CPC - H04S 5/005: {of the pseudo five- or more-channel type, e.g. virtual surround} - G10L 21/038: using band spreading techniques - H04R 3/04: for correcting frequency response - G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing - G10L 21/0364: for improving intelligibility - H03G 5/165: {Equalizers; Volume or gain control in limited frequency bands} - H04S 7/307: {Frequency adjustment, e.g. tone control (H04S7/301 takes precedence)} - H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved - H04S 2400/05: Generation or adaptation of centre channel in multi-channel audio systems - H04S: STEREOPHONIC SYSTEMS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - H03G: CONTROL OF AMPLIFICATION ## Abstract Example audio signal processing apparatuses and methods for processing an input audio signal into an output audio signal are disclosed. An example audio signal processing apparatus comprises a decomposer configured to decompose the input audio signal into a direct audio signal and a diffuse audio signal, a modifier configured to modify the direct audio signal in order to obtain a modified direct audio signal, wherein the modifier comprises a bandwidth extender configured to extend an upper cutoff frequency of a frequency range of the direct audio signal, and a combiner configured to combine the modified direct audio signal with the diffuse audio signal to obtain the output audio signal. ## Claims ### Claim 1 An audio signal processing apparatus for processing an input audio signal into an output audio signal, wherein the audio signal processing apparatus comprises: a decomposer configured to decompose the input audio signal into a direct audio signal and a diffuse audio signal; a modifier configured to modify the direct audio signal in order to obtain a modified direct audio signal, wherein the modifier comprises a bandwidth extender configured to extend an upper cutoff frequency of a frequency range of the direct audio signal; and a combiner configured to combine the modified direct audio signal with the diffuse audio signal to obtain the output audio signal; wherein the bandwidth extender comprises a plurality of bandpass filters configured to extract a plurality of direct audio signal components from the direct audio signal, wherein each direct audio signal component is associated with a corresponding frequency band of a plurality of frequency bands defined by the plurality of bandpass filters, and wherein the bandwidth extender further comprises an energy estimator configured to estimate, for each frequency band of the plurality of frequency bands, the energy of the direct audio signal component associated with the corresponding frequency band; wherein the bandwidth extender further comprises a harmonics generator configured to generate, based on a first direct audio signal component of the plurality of direct audio signal components, an adjustment direct audio signal component containing harmonics of the first direct audio signal component for adjusting a second direct audio signal component of the plurality of direct audio signal components, when the energy of the second direct audio signal component is smaller than a target energy of the second direct audio signal component; wherein the bandwidth extender further comprises a scaler configured to scale the adjustment direct audio signal component for the second direct audio signal component using a scaling factor determined based on the energy of the first direct audio signal component; and wherein the scaler is configured to determine the scaling factor Gh for scaling the adjustment direct audio signal component for the second direct audio signal component based on the following equation: G h = T h E h , wherein Th denotes the target energy of the adjustment direct audio signal component for the second direct audio signal component and Eh denotes the energy of the adjustment direct audio signal component for the second direct audio signal component and wherein Th and Eh are defined by the following equations: Th=Tx−l·Ex, and Eh=β2·El, wherein Ex denotes the energy of the second direct audio signal component, El denotes the energy of the first direct audio signal component, l denotes a leakage factor and β2 denotes a second adjustment factor. ### Claim 2 (depends on 1) The audio signal processing apparatus of claim 1, wherein the decomposer comprises an upmixer configured to upmix the input audio signal into a plurality of upmixed audio signals and wherein the decomposer is configured to generate the direct audio signal and the diffuse audio signal based on the plurality of upmixed audio signals. ### Claim 3 (depends on 2) The audio signal processing apparatus of claim 2, wherein the plurality of upmixed audio signals comprise a center audio signal and wherein the decomposer further comprises a downmixer configured to downmix the center audio signal of the plurality of upmixed audio signals to obtain the direct audio signal. ### Claim 4 (depends on 2) The audio signal processing apparatus of claim 2, wherein the decomposer further comprises an equalizer configured to equalize the plurality of upmixed audio signals. ### Claim 5 (depends on 1) The audio signal processing apparatus of claim 1, wherein the modifier further comprises a transient modifier configured to: detect at least one transient in the direct audio signal; and modify the at least one detected transient in the direct audio signal. ### Claim 6 (depends on 1) The audio signal processing apparatus of claim 1, wherein the modifier further comprises a reverberation filter configured to filter the direct audio signal using different reverberation times for different frequency bands of the direct audio signal, wherein the reverberation time associated with at least one high frequency band of the direct audio signal is larger than the reverberation time associated with at least one low frequency band of the direct audio signal. ### Claim 7 (depends on 6) The audio signal processing apparatus of claim 6, wherein the modifier further comprises a reverberation adjuster configured to adjust the direct audio signal filtered by the reverberation filter based on a third adjustment factor, wherein the third adjustment factor depends on a ratio between the energy of the diffuse audio signal and the direct audio signal provided by the decomposer. ### Claim 8 (depends on 7) The audio signal processing apparatus of claim 7, wherein the reverberation adjuster is configured to adjust the direct audio signal filtered by the reverberation filter based on the third adjustment factor, wherein the ratio between the energy of the modified direct audio signal and the diffuse audio signal of the output audio signal is larger than a predefined fraction of the ratio between the energy of the diffuse audio signal and the direct audio signal provided by the decomposer. ### Claim 9 An audio signal processing apparatus for processing an input audio signal into an output audio signal, wherein the audio signal processing apparatus comprises: at least one processor; and a non-transitory computer-readable storage medium coupled to the at least one processor and storing programming instructions for execution by the at least one processor, wherein the programming instructions instruct the at least one processor to: decompose the input audio signal into a direct audio signal and a diffuse audio signal using a decomposer; modify the direct audio signal to obtain a modified direct audio signal by extending an upper cutoff frequency of a frequency range of the direct audio signal, wherein extending the upper cutoff frequency of the frequency range of the direct audio signal comprises using a bandwidth extender, the bandwidth extender comprising: a plurality of bandpass filters configured to extract a plurality of direct audio signal components from the direct audio signal, wherein each direct audio signal component is associated with a corresponding frequency band of a plurality of frequency bands defined by the plurality of bandpass filters, and wherein the bandwidth extender further comprises an energy estimator configured to estimate, for each frequency band of the plurality of frequency bands, the energy of the direct audio signal component associated with the corresponding frequency band; a harmonics generator configured to generate, based on a first direct audio signal component of the plurality of direct audio signal components, an adjustment direct audio signal component containing harmonics of the first direct audio signal component for adjusting a second direct audio signal component of the plurality of direct audio signal components, when the energy of the second direct audio signal component is smaller than a target energy of the second direct audio signal component; and a scaler configured to scale the adjustment direct audio signal component for the second direct audio signal component using a scaling factor determined based on the energy of the first direct audio signal component, wherein the scaler is configured to determine the scaling factor Gh for scaling the adjustment direct audio signal component for the second direct audio signal component based on the following equation: G h = T h E h , wherein Th denotes the target energy of the adjustment direct audio signal component for the second direct audio signal component and Eh denotes the energy of the adjustment direct audio signal component for the second direct audio signal component and wherein Th and Eh are defined by the following equations: Th=Tx−l·Ex, and Eh=β2·El, wherein Ex denotes the energy of the second direct audio signal component, El denotes the energy of the first direct audio signal component, l denotes a leakage factor and β2 denotes a second adjustment factor; and combine the modified direct audio signal with the diffuse audio signal to obtain the output audio signal. ### Claim 10 (depends on 9) The audio signal processing apparatus of claim 9, wherein the programming instructions further instruct the at least one processor to: upmix the input audio signal into a plurality of upmixed audio signals; and generate the direct audio signal and the diffuse audio signal based on the plurality of upmixed audio signals. ### Claim 11 (depends on 10) The audio signal processing apparatus of claim 10, wherein the plurality of upmixed audio signals comprise a center audio signal and wherein the programming instructions further instruct the at least one processor to: downmix the center audio signal of the plurality of upmixed audio signals to obtain the direct audio signal. ### Claim 12 (depends on 10) The audio signal processing apparatus of claim 10, wherein the programming instructions further instruct the at least one processor to equalize the plurality of upmixed audio signals. ### Claim 13 An audio signal processing apparatus for processing an input audio signal into an output audio signal, wherein the audio signal processing apparatus comprises: a decomposer configured to decompose the input audio signal into a direct audio signal and a diffuse audio signal; a modifier configured to modify the direct audio signal in order to obtain a modified direct audio signal, wherein the modifier comprises a bandwidth extender configured to extend an upper cutoff frequency of a frequency range of the direct audio signal; and a combiner configured to combine the modified direct audio signal with the diffuse audio signal to obtain the output audio signal; wherein the bandwidth extender comprises a plurality of bandpass filters configured to extract a plurality of direct audio signal components from the direct audio signal, wherein each direct audio signal component is associated with a corresponding frequency band of a plurality of frequency bands defined by the plurality of bandpass filters, and wherein the bandwidth extender further comprises an energy estimator configured to estimate, for each frequency band of the plurality of frequency bands, the energy of the direct audio signal component associated with the corresponding frequency band; wherein the bandwidth extender further comprises a harmonics generator configured to generate, based on a first direct audio signal component of the plurality of direct audio signal components, an adjustment direct audio signal component containing harmonics of the first direct audio signal component for adjusting a second direct audio signal component of the plurality of direct audio signal components, when the energy of the second direct audio signal component is smaller than a target energy of the second direct audio signal component; and wherein the target energy of the second direct audio signal component is determined based on the following equation: Tx=β1Tx-1, wherein Tx denotes the target energy of the second direct audio signal component, Tx-1 denotes the target energy of a direct audio signal component associated with the frequency band directly below the frequency band of the second direct audio signal component and β1 denotes a first adjustment factor with β1<1.