# US 10,313,813 B2 · Apparatus and method for sound stage enhancement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,313,813 B2
- Document ID: US10313813B2
- Status: Granted
- Kind: B2
- Application number: 15/963,916
- Filed: 2018-04-26
- Published: 2018-08-30
- Granted: 2019-06-04
- Earliest claimed: 2015-10-27
- Assignees: Ambidio, Inc.
- Applicants: Ambidio, Inc.
- Inventors: Tsai-Yi Wu; Pei-Lun Hsieh
- Web page: https://patentgrep.com/patent/US10313813B2

## CPC

- H04S 3/02: of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
- G10K 11/175: using interference effects; Masking sound
- H03G 5/165: {Equalizers; Volume or gain control in limited frequency bands}
- H04R 5/04: Circuit arrangements, {e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments (combinations of amplifiers H03F3/68; stereophonic systems H04S)}
- H04S 1/007: {in which the audio signals are in digital form (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- H04S 5/005: {of the pseudo five- or more-channel type, e.g. virtual surround}
- H04S 7/307: {Frequency adjustment, e.g. tone control (H04S7/301 takes precedence)}
- G10K 2210/3218: Filters other than the algorithm-related filters
- H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- H04S: STEREOPHONIC SYSTEMS
- G10K: SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR
- H03G: CONTROL OF AMPLIFICATION
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

A method for processing an audio signal is performed at a computing device. The method includes the following steps: receiving a digital stereo audio input signal; extracting localization cues from the digital stereo audio input signal; generating a left-side component and a right-side component from the digital stereo audio input signal, at least partially, in accordance with the localization cues; performing crosstalk cancellation to the left-side component and the right-side component, respectively, to obtain a crosstalk-cancelled left-side component and a crosstalk-cancelled right-side component; and generating a digital stereo audio output signal including the crosstalk-cancelled left-side component and the crosstalk-cancelled right-side component.

## Claims

### Claim 1

A method for processing an audio signal, comprising:
at a computing device having one or more processors, at least two speakers, memory, and a plurality of program modules stored in the memory and to be executed by the one or more processors:
receiving a digital stereo audio input signal;
extracting localization cues from the digital stereo audio input signal;
generating a left-side component and a right-side component from the digital stereo audio input signal, at least partially, in accordance with the localization cues, wherein generating the left-side component and the right-side component from the digital stereo audio input signal further includes:
extracting a middle component, an initial left-side component and an initial right-side component from the digital stereo audio input signal;
panning the middle component into the initial left-side component and the initial right-side component, respectively, in accordance with the localization cues to obtain an intermediate left-side component and an intermediate right-side component; and
upmixing a first copy of the middle component with the intermediate left-side component and a second copy of the middle component with the intermediate right-side component, respectively, to obtain the left-side component and the right-side component, wherein there is a predefined time interval between the first and second copies of the middle component;
performing crosstalk cancellation to the left-side component and the right-side component, respectively, to obtain a crosstalk-cancelled left-side component and a crosstalk-cancelled right-side component; and
generating a digital stereo audio output signal including the crosstalk-cancelled left-side component and the crosstalk-cancelled right-side component.

### Claim 2 (depends on 1)

The method of claim 1, wherein the middle component, the intermediate left-side component and the intermediate right-side component correspond to a center channel, a left surround signal and a right surround channel of a multi-channel audio system, respectively.

### Claim 3 (depends on 1)

The method of claim 1, wherein the step of panning the middle component into the initial left-side component and the initial right-side component, respectively, in accordance with the localization cues to obtain an intermediate left-side component and an intermediate right-side component further includes:
receiving a center channel, a left surround channel and a right surround channel of a multi-channel audio system;
combining the center channel with the middle component; and
combining the left surround channel and the right surround channel with the intermediate left-side component and the intermediate right-side component, respectively.

### Claim 4 (depends on 1)

The method of claim 1, wherein the digital stereo audio input signal further includes a left signal and a right signal, and the step of extracting a middle component, an initial left-side component and an initial right-side component from the digital stereo audio input signal further includes:
transforming the left signal and the right signal from a time domain to a frequency domain using discrete short-time Fourier transform;
defining the middle component in the frequency domain as one of the left signal and the right signal in the frequency domain having a smaller magnitude for a predefined frequency in the frequency domain;
transforming the middle component from the frequency domain back to the time domain using discrete short-time inverse Fourier transform; and
generating the initial left-side component and the initial right-side component by subtracting the middle component in the time domain from the left signal and the right signal, respectively.

### Claim 5 (depends on 1)

The method of claim 1, wherein the digital stereo audio input signal further includes a left signal and a right signal, the step of panning the middle component into the initial left-side component and the initial right-side component, respectively, in accordance with the localization cues to obtain an intermediate left-side component and an intermediate right-side component further includes:
applying a left amplitude panning parameter and a right amplitude panning parameter to the middle component, respectively, to obtain an initial mid-left component and an initial mid-right component;
determining an interaural time difference (ITD) and an interaural intensity difference (IID) between the left signal and the right signal;
applying the ITD and the IID to at least one of the initial mid-left component and the initial mid-right component to obtain an intermediate mid-left component and an intermediate mid-right component; and
combining the initial left-side component with the intermediate mid-left component and the initial right-side component with the intermediate mid-right component, respectively, to obtain the intermediate left-side component and the intermediate right-side component.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
determining an angle of a sound source associated with the digital stereo audio input signal;
calculating the ITD and the IID in accordance with the angle of the sound source;
updating the initial mid-right component by applying a time shift of the ITD and an amplitude multiplication of the IID to the initial mid-right component; and
combining the updated initial mid-right component into the initial mid-left component as the intermediate mid-left component.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
before performing crosstalk cancellation to the left-side component and the right-side component:
performing equalization to the left-side component and the right-side component using a first bandpass filter to obtain a bandpass-filtered left-side component and a bandpass-filtered right-side component, respectively;
generating a left-side residual component and a right-side residual component by subtracting the left-side component and the right-side component from the bandpass-filtered left-side component and the bandpass-filtered right-side component, respectively; and
after performing crosstalk cancellation to the left-side component and the right-side component:
combining the left-side residual component and the right-side residual component into the crosstalk-cancelled left-side component and the crosstalk-cancelled right-side component, respectively.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
after performing equalization to the left-side component and the right-side component using the first bandpass filter:
removing a predefined frequency band from the left-side component and the right-side component using a second bandpass filter.

### Claim 9 (depends on 7)

The method of claim 7, further comprising:
after performing equalization to the left-side component and the right-side component using the first bandpass filter:
performing first dynamic range compression to the left-side component and the right-side component to highlight a predefined frequency band with respect to other frequencies.

### Claim 10 (depends on 7)

The method of claim 7, further comprising:
after combining the left-side residual component and the right-side residual component into the crosstalk-cancelled left-side component and the crosstalk-cancelled right-side component, respectively:
performing second dynamic range compression to the crosstalk-cancelled left-side component and the crosstalk-cancelled right-side component to preserve the localization cues in the digital stereo audio output signal.

### Claim 11

A computing device for processing an audio signal, comprising:
one or more processors;
memory; and
a plurality of program modules stored in the memory and to be executed by the one or more processors, wherein the plurality of program modules, when executed by the one or more processors, cause the computing device to perform a plurality of steps including:
receiving a digital stereo audio input signal;
extracting localization cues from the digital stereo audio input signal;
generating a left-side component and a right-side component from the digital stereo audio input signal, at least partially, in accordance with the localization cues, wherein generating the left-side component and the right-side component from the digital stereo audio input signal further includes:
extracting a middle component, an initial left-side component and an initial right-side component from the digital stereo audio input signal;
panning the middle component into the initial left-side component and the initial right-side component, respectively, in accordance with the localization cues to obtain an intermediate left-side component and an intermediate right-side component; and
upmixing a first copy of the middle component with the intermediate left-side component and a second copy of the middle component with the intermediate right-side component, respectively, to obtain the left-side component and the right-side component, wherein there is a predefined time interval between the first and second copies of the middle component;
performing crosstalk cancellation to the left-side component and the right-side component, respectively, to obtain a crosstalk-cancelled left-side component and a crosstalk-cancelled right-side component; and
generating a digital stereo audio output signal including the crosstalk-cancelled left-side component and the crosstalk-cancelled right-side component.

### Claim 12 (depends on 11)

The computing device of claim 11, wherein the middle component, the intermediate left-side component and the intermediate right-side component correspond to a center channel, a left surround signal and a right surround channel of a multi-channel audio system, respectively.

### Claim 13 (depends on 11)

The computing device of claim 11, wherein the step of panning the middle component into the initial left-side component and the initial right-side component, respectively, in accordance with the localization cues to obtain an intermediate left-side component and an intermediate right-side component further includes:
receiving a center channel, a left surround channel and a right surround channel of a multi-channel audio system;
combining the center channel with the middle component; and
combining the left surround channel and the right surround channel with the intermediate left-side component and the intermediate right-side component, respectively.

### Claim 14 (depends on 11)

The computing device of claim 11, wherein the digital stereo audio input signal further includes a left signal and a right signal, and the step of extracting a middle component, an initial left-side component and an initial right-side component from the digital stereo audio input signal further includes:
transforming the left signal and the right signal from a time domain to a frequency domain using discrete short-time Fourier transform;
defining the middle component in the frequency domain as one of the left signal and the right signal in the frequency domain having a smaller magnitude for a predefined frequency in the frequency domain;
transforming the middle component from the frequency domain back to the time domain using discrete short-time inverse Fourier transform; and
generating the initial left-side component and the initial right-side component by subtracting the middle component in the time domain from the left signal and the right signal, respectively.

### Claim 15 (depends on 11)

The computing device of claim 11, wherein the digital stereo audio input signal further includes a left signal and a right signal, the step of panning the middle component into the initial left-side component and the initial right-side component, respectively, in accordance with the localization cues to obtain an intermediate left-side component and an intermediate right-side component further includes:
applying a left amplitude panning parameter and a right amplitude panning parameter to the middle component, respectively, to obtain an initial mid-left component and an initial mid-right component;
determining an interaural time difference (ITD) and an interaural intensity difference (IID) between the left signal and the right signal;
applying the ITD and the IID to at least one of the initial mid-left component and the initial mid-right component to obtain an intermediate mid-left component and an intermediate mid-right component; and
combining the initial left-side component with the intermediate mid-left component and the initial right-side component with the intermediate mid-right component, respectively, to obtain the intermediate left-side component and the intermediate right-side component.

### Claim 16 (depends on 11)

The computing device of claim 11, wherein the plurality of steps further include:
before performing crosstalk cancellation to the left-side component and the right-side component:
performing equalization to the left-side component and the right-side component using a first bandpass filter to obtain a bandpass-filtered left-side component and a bandpass-filtered right-side component, respectively;
generating a left-side residual component and a right-side residual component by subtracting the left-side component and the right-side component from the bandpass-filtered left-side component and the bandpass-filtered right-side component, respectively; and
after performing crosstalk cancellation to the left-side component and the right-side component:
combining the left-side residual component and the right-side residual component into the crosstalk-cancelled left-side component and the crosstalk-cancelled right-side component, respectively.

### Claim 17

A non-transitory computer readable storage medium storing a computer program product in connection with a computing device having one or more processors for processing an audio signal, the computer program product including a plurality of program modules that, when executed by the one or more processors, cause the computing device to perform a plurality of steps including:
receiving a digital stereo audio input signal;
extracting localization cues from the digital stereo audio input signal;
generating a left-side component and a right-side component from the digital stereo audio input signal, at least partially, in accordance with the localization cues, wherein generating the left-side component and the right-side component from the digital stereo audio input signal further includes:
extracting a middle component, an initial left-side component and an initial right-side component from the digital stereo audio input signal;
panning the middle component into the initial left-side component and the initial right-side component, respectively, in accordance with the localization cues to obtain an intermediate left-side component and an intermediate right-side component; and
upmixing a first copy of the middle component with the intermediate left-side component and a second copy of the middle component with the intermediate right-side component, respectively, to obtain the left-side component and the right-side component, wherein there is a predefined time interval between the first and second copies of the middle component;
performing crosstalk cancellation to the left-side component and the right-side component, respectively, to obtain a crosstalk-cancelled left-side component and a crosstalk-cancelled right-side component; and
generating a digital stereo audio output signal including the crosstalk-cancelled left-side component and the crosstalk-cancelled right-side component.

### Claim 18 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the plurality of steps further include:
before performing crosstalk cancellation to the left-side component and the right-side component:
performing equalization to the left-side component and the right-side component using a first bandpass filter to obtain a bandpass-filtered left-side component and a bandpass-filtered right-side component, respectively;
generating a left-side residual component and a right-side residual component by subtracting the left-side component and the right-side component from the bandpass-filtered left-side component and the bandpass-filtered right-side component, respectively; and
after performing crosstalk cancellation to the left-side component and the right-side component:
combining the left-side residual component and the right-side residual component into the crosstalk-cancelled left-side component and the crosstalk-cancelled right-side component, respectively.