Systems and Methods for Brain-Informed Speech Separation

A system comprising:

• at least one microphone to obtain a combined sound signal for signals combined from multiple sound sources in an area in which a person is located;
• one or more neural sensors to obtain neural signals for the person, the neural signals being indicative of one or more target sound sources, from the multiple sound sources, the person is attentive to;
• and
• a controller in communication with the at least one microphone and the one or more neural sensors, the controller configured to:
• determine a separation filter based, at least in part, on the neural signals obtained for the person;
• and
• apply the separation filter to a representation of the combined sound signal to derive a resultant separated signal representation associated with sound from the one or more target sound sources the person is attentive to;
• wherein the combined sound signal comprises sound components corresponding to multiple receiving channels, and wherein the controller configured to determine the separation filter is configured to:
• apply multiple encoders to the sound components corresponding to the multiple receiving channels, with each of the encoders applied to each of the sound components;
• combine, for each of the multiple receiving channels, output components of the multiple encoders associated with respective ones of the multiple receiving channels;
• and
• derive estimated separation functions based on the combined output components for each of the multiple receiving channels, each of the derived estimated separation functions configured to separate the combined output components for each of the multiple receiving channels into separated sound components associated with groups of the multiple sound sources.

Non-transitory computer readable media comprising computer instructions executable on a processor-based device to:

• obtain a combined sound signal for signals combined from multiple sound sources in an area in which a person is located;
• obtain neural signals for the person, the neural signals being indicative of one or more target sound sources, from the multiple sound sources, the person is attentive to;
• determine a separation filter based, at least in part, on the neural signals obtained for the person;
• and
• apply the separation filter to a representation of the combined sound signal to derive a resultant separated signal representation associated with sound from the one or more target sound sources the person is attentive to;
• wherein the combined sound signal comprises sound components corresponding to multiple receiving channels, and wherein the computer instructions to determine the separation filter comprise one or more computer instructions to:
• apply multiple encoders to the sound components corresponding to the multiple receiving channels, with each of the encoders applied to each of the sound components;
• for each of the multiple receiving channels, combine output components of the multiple encoders associated with respective ones of the multiple receiving channels;
• and
• derive estimated separation functions based on the combined output components for each of the multiple receiving channels, each of the derived estimated separation functions configured to separate the combined output components for each of the multiple receiving channels into separated sound components associated with groups of the multiple sound sources.