Methods and Systems for End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

A method for transforming input audio signals, comprising the steps of:

• using a module for transforming an input audio signal of the input audio signals, such that the input audio signal includes a mixture of one or more target audio signals, wherein the module transforms the input audio signal, to obtain target audio signal estimates;
• using a spectrogram estimator of the model, to extract an audio feature sequence from the input audio signal, and process the audio feature sequence to output a set of estimated spectrograms, wherein the set of estimated spectrograms includes an estimated spectrogram for each target audio signal;
• using a spectrogram refinement module of the module to process the set of estimated spectrograms and the audio feature sequence, to output a set of refined spectrograms, such that the set of refined spectrograms includes a refined spectrogram for each target audio signal, and wherein using the spectrogram refinement module is based on an iterative reconstruction algorithm;
• using a signal refinement module of the module to process the set of refined spectrograms for the one or more target audio signals, to obtain target audio signal estimates, such that there is an target audio signal estimate for each target audio signal;
• and
• outputting the target audio signal estimates, wherein parameters of the stored module are trained using training data by minimizing an error using an optimizer stored in a memory, wherein the error includes one or more of an error on the set of refined spectrograms, an error including a consistency measurement on the set of refined spectrograms, or an error on the target audio signal estimates, and wherein the steps are performed by a processor in communication with an output device and the memory having stored executable instructions, such that the module is stored in the memory.

An audio signal processing system for transforming input audio signals, the audio signal processing system comprising:

• a memory including stored executable instructions and a stored module, such that the stored module transforms the input audio signal of the input audio signals to obtain target audio signal estimates, wherein input audio signal includes a mixture of one or more target audio signals;
• an input interface to receive the input audio signal;
• a processor in communication with the memory and the input interface, wherein the processor implements steps of the stored module by
• a spectrogram estimator of the stored module to extract an audio feature sequence from the input audio signal, and process the audio feature sequence to output a set of estimated spectrograms, wherein the set of estimated spectrograms includes an estimated spectrogram for each target audio signal;
• a spectrogram refinement module of the stored module to process the set of estimated spectrograms and the audio feature sequence, to output a set of refined spectrograms, such that the set of refined spectrograms includes a refined spectrogram for each target audio signal, and wherein using the spectrogram refinement module is based on an iterative reconstruction algorithm;
• a signal refinement module of the stored module to process the set of refined spectrograms for the one or more target audio signals, to obtain target audio signal estimates, such that there is an target audio signal estimate for each target audio signal;
• and
• an output interface to output the target audio signal estimates,
• wherein parameters of the stored module are trained using training data by minimizing an error using an optimizer stored in the memory, wherein the error includes an error on the set of refined spectrograms or an error on the target audio signal estimates or both.

An audio signal processing system for transforming input audio signals, the audio signal processing comprising:

• a sound detecting device configured to acquire the input audio signal of the input audio signals from an environment;
• a signal input interface device configured to receive and transmit the input audio signal, wherein the input audio signal includes a mixture of one or more target audio signals;
• an audio signal processing device configured to process the input audio signal, wherein the audio signal processing device comprises:
• a hardware processor configured to connected to a memory, the memory configured to input/output data, wherein the hardware processor executes the steps of a module:
• accessing the module stored in the memory, such that the stored module transforms the input audio signal to obtain target audio signal estimates;
• a spectrogram estimator of the stored module, to extract an audio feature sequence from the input audio signal, and process the audio feature sequence to output a set of estimated spectrograms, wherein the set of estimated spectrograms includes an estimated spectrogram for each target audio signal;
• a spectrogram refinement module of the stored module to process the set of estimated spectrograms and the audio feature sequence, to output a set of refined spectrograms, such that the set of refined spectrograms includes a refined spectrogram for each target audio signal, and wherein using the spectrogram refinement module is based on an iterative reconstruction algorithm;
• a signal refinement module of the stored module to process the set of refined spectrograms for the one or more target audio signals, to obtain target audio signal estimates, such that there is an target audio signal estimate for each target audio signal;
• and
• an output interface to output the target audio signal estimates,
• wherein parameters of the stored module are trained using training data by minimizing an error using an optimizer stored in the memory, wherein the error includes one or more of an error on the set of refined spectrograms, an error including a consistency measurement on the set of refined spectrograms, or an error on the target audio signal estimates.

An audio signal processing system for transforming an input audio signal from a single-channel signal, the audio signal processing system includes an end-to-end approach for single-channel speaker-independent multi-speaker speech separation, the audio signal processing system comprising:

• a memory including stored executable instructions and a stored module, such that the stored module transforms the input audio signal to obtain target audio signal estimates;
• a processor in communication with the memory, wherein the processor implements steps of the stored module by
• a spectrogram estimator of the stored module receives the input audio signal from the single-channel signal, to extract an audio feature sequence, and process the audio feature sequence to output a set of estimated spectrograms, wherein the set of estimated spectrograms includes an estimated spectrogram for each target audio signal;
• a spectrogram refinement module of the stored module to process the set of estimated spectrograms and the audio feature sequence, to output a set of refined spectrograms, such that the set of refined spectrograms includes a refined spectrogram for each target audio signal, and wherein using the spectrogram refinement module is based on an iterative reconstruction algorithm;
• a signal refinement module of the stored module to process the set of refined spectrograms for the one or more target audio signals, to obtain target audio signal estimates, such that there is an target audio signal estimate for each target audio signal;
• and
• an output interface to output the target audio signal estimates, wherein parameters of the stored module are trained using training data by minimizing an error using an optimizer stored in the memory, wherein the error includes one or more of an error on the set of refined spectrograms, an error including a consistency measurement on the set of refined spectrograms, or an error on the target audio signal estimates.