Real-time audio source separation using deep neural networks
AssigneeRed Pill VR, Inc
InventorsAlejandro Koretzky, Karthiek Reddy Bokka, Naveen Sasalu Rajashekharappa
Life of this patent
Estimating the term…
Abstract
Methods and systems for audio source separation in real-time are described. In an embodiment, the present disclosure describes reading and decoding an audio source into PCM samples, fragmenting Pulse Code Modulation (PCM) samples into fragments, transforming fragments into spectrograms, performing audio source separation using a deep neural network (DNN) to generate an estimated magnitude spectrogram of the component(s) of the audio source, reconstructing the estimated time domain component signals, and streaming the component signals to a playback engine. In an embodiment, a semantic equalizer graphical user allows for real-time mixing of individual component signals.
Claims18
A method comprising:
- reading an audio source, wherein the audio source represents a combined audio signal that comprises a plurality of component signals;
- generating, based on the audio source, a plurality of pulse-code modulation (PCM) samples;
- fragmenting the PCM samples into a plurality of PCM sample fragments;
- for each particular PCM sample fragment of the plurality of PCM sample fragments:
- transforming the particular PCM sample fragment into a spectrogram;
- processing the spectrogram into one or more spectrogram fragments;
- inputting the one or more spectrogram fragments into a deep neural network;
- generating, by the deep neural network, one or more binary mask fragments wherein each binary mask fragment of the one or more binary mask fragments corresponds to one or more component signals of the plurality of component signals;
- using the one or more binary mask fragments to estimate the plurality of component signals;
- wherein the method is performed using one or more processors.
One or more non-transitory computer-readable media storing instructions, which when executed by one or more processors cause:
- reading an audio source, wherein the audio source represents a combined audio signal that comprises a plurality of component signals;
- generating, based on the audio source, a plurality of pulse-code modulation (PCM) samples;
- fragmenting the PCM samples into a plurality of PCM sample fragments;
- for each particular PCM sample fragment of the plurality of PCM sample fragments:
- transforming the particular PCM sample fragment into a spectrogram;
- processing the spectrogram into one or more spectrogram fragments;
- inputting the one or more spectrogram fragments into a deep neural network;
- generating, by the deep neural network, one or more binary mask fragments corresponding to the plurality of component signals;
- and
- using the one or more binary mask fragments to estimate the plurality of component signals.
Dependencies are inferred from the claim text.