Method and system for instrument separating and reproducing for mixture audio source
AssigneeHarman International Industries, Incorporated
InventorsJianwen Zheng, Hongfei Zhou
Life of this patent
Estimating the term…
Abstract
A method and a system for instrument separating and reproducing for a mixture audio source is provided. The method and/or the system includes inputting selected music into an instrument separation model for extracting features therefrom, determining audio source signals of multiple channels for the separation of all instruments, each channel containing sound of one instrument, and transmitting the signals of the different channels to multiple speakers placed at designated positions for playing, which can reproduce or recreate an immersive sound field listening experience for users.
Claims25
A method for instrument separating and reproducing for a mixture audio source, comprising:
- obtaining a mixture audio source spectrogram based on the mixture audio source, wherein the mixture audio source comprises sound of at least one instrument;
- using an instrument separation model to sequentially obtain an instrument feature mask of each of the at least one instrument from the mixture audio source;
- obtaining an instrument spectrogram of each of the at least one instrument based on the instrument feature mask for each of the at least one instrument;
- determining an instrument audio source of the each of the at least one instrument based on the instrument spectrogram;
- and
- respectively feeding the respective instrument audio source of the at least one instrument to at least one speaker, and reproducing the respective instrument audio source of the at least one instrument accordingly by the at least one speaker,
- wherein respectively feeding the respective instrument audio source of the at least one instrument to at least one speaker comprises modulating the respective instrument audio source of the at least one instrument into at least one corresponding broadcast audio signal and broadcasting the corresponding broadcast audio signal to the at least one speaker in a form of multi channels, and correspondingly demodulating the corresponding instrument audio source of the at least one instrument by the at least one speaker.
A non-transitory computer-readable medium including instructions that, when executed by a processor, perform the following steps including:
- obtaining a mixture audio source spectrogram based on a mixture audio source, wherein the mixture audio source comprises sound of at least one instrument;
- using an instrument separation model to sequentially obtain an instrument feature mask of each of the at least one instrument from the mixture audio source;
- obtaining an instrument spectrogram of each of the at least one instrument based on the instrument feature mask for each of the at least one instrument;
- determining an instrument audio source of each of the at least one instrument based on the instrument spectrogram;
- and
- respectively feeding the instrument audio sources of the at least one instrument to at least one speaker for reproducing,
- wherein respectively feeding the respective instrument audio source of the at least one instrument to at least one speaker comprises modulating the respective instrument audio source of the at least one instrument into at least one corresponding broadcast audio signal and broadcasting the at least one corresponding broadcast audio signal to the at least one speaker in form of multi channels.
A system for instrument separating and reproducing for a mixture audio source, comprising:
- a spectrogram conversion module configured to obtain a mixture audio source spectrogram based on the mixture audio source, wherein the mixture audio source comprises sound of at least one instrument;
- an instrument separation module comprising an instrument separation model, wherein the instrument separation model is configured to sequentially obtain an instrument feature mask of each of the at least one instrument from the mixture audio source;
- an instrument extraction module configured to obtain an instrument spectrogram of each of the at least one instrument based on the instrument feature mask for the each of the at least one instrument;
- and
- an instrument audio source rebuilding module configured to determine an instrument audio source for each of the at least one instrument based on the instrument spectrogram, wherein the instrument audio source for the at least one instrument is respectively transmitted to at least one speaker and is reproduced by the at least one speaker,
- wherein respectively feeding the respective instrument audio source of the at least one instrument to at least one speaker comprises modulating the respective instrument audio source of the at least one instrument into at least one corresponding broadcast audio signal and broadcasting the at least one corresponding audio signal to the at least one speaker in form of multi channels, and correspondingly demodulating the corresponding instrument audio source of the at least one instrument by the at least one speaker.
Dependencies are inferred from the claim text.