# US 2013/0297296 A1 · Source separation by independent component analysis in conjunction with source direction information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0297296 A1
- Document ID: US20130297296A1
- Status: Published application
- Kind: A1
- Application number: 13/464,828
- Filed: 2012-05-04
- Published: 2013-11-07
- Earliest claimed: 2012-05-04
- Assignees: Sony Computer Entertainment Inc.
- Inventors: Jaekwon Yoo; Ruxin Chen
- Web page: https://patentgrep.com/patent/US20130297296A1

## CPC

- G10L 21/0272: Voice signal separating
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Methods and apparatus for signal processing are disclosed. Source separation can be performed to extract source signals from mixtures of source signals by way of independent component analysis. Source direction information is utilized in the separation process, and independent component analysis techniques described herein use multivariate probability density functions to preserve the alignment of frequency bins in the source separation process. It is emphasized that this abstract is provided to comply with the rules requiring an abstract that will allow a searcher or other reader to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

## Claims

### Claim 1

A method of processing signals with a signal processing device, comprising:
receiving a plurality of time domain mixed signals in a signal processing device, each time domain mixed signal including a mixture of original source signals;
performing a Fourier-related transform on each time domain mixed signal with the signal processing device to generate time-frequency domain mixed signals corresponding to the time domain mixed signals; and
performing independent component analysis on the time-frequency domain mixed signals to generate at least one estimated source signal corresponding to at least one of the original source signals,
wherein the independent component analysis is performed in conjunction with a direction constraint based on a known direction of an original source signal with respect to a sensor array that detected the time domain mixed signals, and
wherein the independent component analysis uses a multivariate probability density function to preserve the alignment of frequency bins in the at least one estimated source signal.

### Claim 2 (depends on 1)

The method of claim 1, wherein the mixed signals are audio signals.

### Claim 3 (depends on 1)

The method of claim 1, wherein the mixed signals include at least one speech source signal, and the at least one estimated source signal corresponds to said at least one speech signal.

### Claim 4 (depends on 1)

The method of claim 1, wherein the multivariate probability density function is a mixed multivariate probability density function that is a weighted mixture of component multivariate probability density functions of frequency bins corresponding to different source signals and/or different time segments.

### Claim 5 (depends on 1)

The method of claim 1, wherein the multivariate probability density function is a mixed multivariate probability density function that is a weighted mixture of component multivariate probability density functions of frequency bins corresponding to different source signals and/or different time segments, wherein said performing independent component analysis comprises utilizing an expectation maximization algorithm to estimate the parameters of the component multivariate probability density functions.

### Claim 6 (depends on 1)

The method of claim 1, wherein the direction constraint is based on a phase difference among mixing filters, each mixing filter modeling a mixing process of the original source signals at each sensor in the sensor array.

### Claim 7 (depends on 1)

The method of claim 1, wherein performing the independent component analysis includes use of a cost function that includes both a function corresponding to unconstrained independent component analysis and a function corresponding to the direction constraint, wherein the direction constraint is chosen to make demixing filters of a demixing matrix have a flat spectral response.

### Claim 8 (depends on 1)

The method of claim 1, wherein said performing a Fourier-related transform comprises performing a short time Fourier transform (STFT) over a plurality of discrete time segments.

### Claim 9 (depends on 1)

The method of claim 1, wherein said performing independent component analysis includes utilizing pre-trained eigenvectors of clean signals in an estimation of the parameters of the component probability density function.

### Claim 10 (depends on 1)

The method of claim 1, wherein said performing independent component analysis further comprises utilizing pre-trained eigenvectors of music and noise.

### Claim 11 (depends on 1)

The method of claim 1, wherein said performing independent component analysis further comprises training eigenvectors with run-time data.

### Claim 12 (depends on 1)

The method of claim 1, further comprising converting the mixed signals into digital form with an analog to digital converter before said performing a Fourier-related transform.

### Claim 13 (depends on 1)

The method of claim 1, further comprising performing an inverse STFT on the at least one estimated time-frequency domain source signal to produce at least one estimated time domain source signal corresponding to an original time domain source signal.

### Claim 14 (depends on 1)

The method of claim 1, wherein the multivariate probability density function includes a spherical distribution.

### Claim 15 (depends on 1)

The method of claim 1, wherein the multivariate probability density function includes a Laplacian distribution.

### Claim 16 (depends on 1)

The method of claim 1, wherein the multivariate probability density function includes a super-Gaussian distribution.

### Claim 17 (depends on 1)

The method of claim 1, wherein the multivariate probability density function includes a multivariate generalized Gaussian distribution.

### Claim 18 (depends on 1)

The method of claim 1, wherein the multivariate probability density function is a mixed multivariate probability density function, wherein said mixed multivariate probability density function is a weighted mixture of component probability density functions of frequency bins corresponding to different sources.

### Claim 19 (depends on 1)

The method of claim 1, wherein the multivariate probability density function is a mixed multivariate probability density function, wherein said mixed multivariate probability density function is a weighted mixture of component probability density functions of frequency bins corresponding to different time segments

### Claim 20 (depends on 1)

The method of claim 1, wherein the sensor array is a microphone array, and the method further comprises observing the time domain mixed signals with the microphone array before receiving the time domain mixed signals in a signal processing device.

### Claim 21

A signal processing device comprising:
a processor;
a memory; and
computer coded instructions embodied in the memory and executable by the processor, wherein the instructions are configured to implement a method of signal processing comprising:
receiving a plurality of time domain mixed signals, each time domain mixed signal including a mixture of original source signals;
performing a Fourier-related transform on each time domain mixed signal to generate time-frequency domain mixed signals corresponding to the time domain mixed signals; and
performing independent component analysis on the time-frequency domain mixed signals to generate at least one estimated source signal corresponding to at least one of the original source signals,
wherein the independent component analysis is performed in conjunction with a direction constraint based on a known direction, with respect to a sensor array that detected the time domain mixed, of an original source signal signals, and
the independent component analysis uses a multivariate probability density function to preserve the alignment of frequency bins in the at least one estimated source signal.

### Claim 22 (depends on 21)

The device of claim 21, further comprising the sensor array.

### Claim 23 (depends on 21)

The device of claim 21, wherein the sensor array is a microphone array.

### Claim 24 (depends on 21)

The device of claim 21, wherein the mixed signals include at least one speech source signal, and the at least one estimated source signal corresponds to said at least one speech signal.

### Claim 25 (depends on 21)

The device of claim 21, wherein the multivariate probability density function is a mixed multivariate probability density function that is a weighted mixture of component multivariate probability density functions of frequency bins corresponding to different source signals and/or different time segments.

### Claim 26 (depends on 21)

The device of claim 21, wherein the direction constraint is based on a phase difference among mixing filters, each mixing filter modeling a mixing process of the original source signals at each sensor in the sensor array.

### Claim 27 (depends on 21)

The device of claim 21, wherein said performing a Fourier-related transform comprises performing a short time Fourier transform (STFT) over a plurality of discrete time segments.

### Claim 28 (depends on 21)

The device of claim 21, wherein the multivariate probability density function is a mixed multivariate probability density function that is a weighted mixture of component multivariate probability density functions of frequency bins corresponding to different source signals and/or different time segments, wherein said performing independent component analysis comprises utilizing an expectation maximization algorithm to estimate the parameters of the component multivariate probability density functions.

### Claim 29 (depends on 21)

The device of claim 21, wherein the multivariate probability density function is a mixed multivariate probability density function that is a weighted mixture of component multivariate probability density functions of frequency bins corresponding to different source signals and/or different time segments, wherein said performing independent component analysis comprises utilizing pre-trained eigenvectors of a clean signal in an estimation of the parameters of the component probability density functions.

### Claim 30 (depends on 29)

The device of claim 29, wherein said performing independent component analysis further comprises utilizing pre-trained eigenvectors of music and noise.

### Claim 31 (depends on 29)

The device of claim 29, wherein said performing independent component analysis further comprises training eigenvectors with run-time data.

### Claim 32 (depends on 21)

The device of claim 21, further comprising an analog to digital converter, wherein said method of signal processing further comprises converting the mixed signals into digital form with the analog to digital converter before said performing a Fourier-related transform.

### Claim 33 (depends on 21)

The device of claim 21, the method further comprising performing an inverse STFT on the estimated time-frequency domain source signals to produce estimated time domain source signals corresponding to original time domain source signals.

### Claim 34 (depends on 21)

The device of claim 21, wherein the multivariate probability density function includes a spherical distribution.

### Claim 35 (depends on 34)

The device of claim 34, wherein the multivariate probability density function includes a Laplacian distribution.

### Claim 36 (depends on 34)

The device of claim 34, wherein the multivariate probability density function includes a super-Gaussian distribution.

### Claim 37 (depends on 21)

The device of claim 21, wherein the multivariate probability density function includes a multivariate generalized Gaussian distribution.

### Claim 38 (depends on 21)

The device of claim 21, wherein said mixed multivariate probability density function is a weighted mixture of component probability density functions of frequency bins corresponding to different sources.

### Claim 39 (depends on 21)

The device of claim 21, wherein said mixed multivariate probability density function is a weighted mixture of component probability density functions of frequency bins corresponding to different time segments

### Claim 40

A computer program product comprising a non-transitory computer-readable medium having computer-readable program code embodied in the medium, the program code operable to perform signal processing operations comprising:
receiving a plurality of time domain mixed signals, each time domain mixed signal including a mixture of original source signals;
performing a Fourier-related transform on each time domain mixed signal to generate time-frequency domain mixed signals corresponding to the time domain mixed signals; and
performing independent component analysis on the time-frequency domain mixed signals to generate at least one estimated source signal corresponding to at least one of the original source signals,
wherein the independent component analysis is performed in conjunction with a direction constraint based on a known direction, with respect to a sensor array that detected the time domain mixed signals, of an original source signal, and
the independent component analysis uses a multivariate probability density function to preserve the alignment of frequency bins in the at least one estimated source signal.
