Method and apparatus for task-driven speech separation by leveraging speaker distance information

An apparatus comprising:

• at least one memory configured to store program code;
• and
• at least one processor configured to read the program code and operate as instructed by the program code, the program code including:
• receiving code configured to cause the at least one processor to receive a mixture signal comprising at least a first speaker, a second speaker, and background noise, the first speaker having a first distance to a microphone that outputs the mixture signal, the second speaker having a second distance to the microphone;
• training code configured to cause the at least one processor to train one or more neural networks to output a target channel and an interference channel, the training code comprising:
• converting code configured to cause the at least one processor to convert the mixture signal to a short-time Fourier transform (STFT);
• extracting code configured to cause the at least one processor to extract, from an embedding layer that receives the first distance and the second distance, a task ID associated with one of the first speaker and the second speaker as a target speaker;
• concatenating code configured to cause the at least one processor to concatenate the task ID with the mixture signal converted to the STFT to generate a concatenated signal;
• first inputting code configured to cause the at least one processor to input, into the one or more neural networks, the concatenated signal to output the target channel and the interference channel;
• determining code configured to cause the at least one processor to determine a loss function based on the first distance and the second distance;
• and
• updating code configured to cause the at least one processor to update one or more parameters of the neural network based on the loss function such that outputting the target channel and the interference channel minimizes the loss function.

A non-transitory computer readable medium, having instructions stored therein, which when executed by a processor cause the method to execute a method comprising:

• receiving a mixture signal comprising at least a first speaker, a second speaker, and background noise, the first speaker having a first distance to a microphone that outputs the mixture signal, the second speaker having a second distance to the microphone;
• training one or more neural networks to output a target channel and an interference channel by:
• converting the mixture signal to a short-time Fourier transform (STFT);
• extracting, from an embedding layer that receives the first distance and the second distance, a task ID associated with one of the first speaker and the second speaker as a target speaker;
• concatenating the task ID with the mixture signal converted to the STFT to generate a concatenated signal;
• inputting, into the one or more neural networks, the concatenated signal to output the target channel and the interference channel;
• determining a loss function based on the first distance and the second distance;
• and
• updating one or more parameters of the neural network based on the loss function such that outputting the target channel and the interference channel minimizes the loss function.