Back to results
US 12,057,131 B2Granted
Deep learning segmentation of audio using magnitude spectrogram
AssigneeAudioshake, Inc.
InventorLuke Miner
Life of this patent
Estimating the term…
Abstract
A method, system, and computer readable medium for decomposing an audio signal into different isolated sources. The techniques and mechanisms convert an audio signal into K input spectrogram fragments. The fragments are sent into a deep neural network to isolate for different sources. The isolated fragments are then combined to form full isolated source audio signals.
Claims20
A method for isolating a source from an audio signal, the method comprising:
- transforming an audio file into a complex spectrogram;
- decomposing the complex spectrogram into a magnitude spectrogram and a phase spectrogram;
- splitting the magnitude spectrogram into K small fragments along the time dimension;
- sending each fragment in the K small fragments through a convolutional deep neural network to produce K small fragment outputs, wherein the convolutional deep neural network includes convolutional layers augmented with an attention layer to help the convolutional deep neural network find the most informative areas and reduce artifacts in each of the K small fragments, wherein the attention layer informs the convolutional deep neural network of important pixels;
- producing a source mask based on the K small fragment outputs, wherein the source mask corresponds to an audio source targeted to be isolated;
- multiplying the source mask with the magnitude spectrogram to create a new magnitude spectrogram corresponding to the source.
3 more elements
A system for isolating a source from an audio signal, the system comprising:
- a processor;
- and
- memory storing instructions to cause the processor to execute a method, the method comprising:
- transforming an audio file into a complex spectrogram;
- decomposing the complex spectrogram into a magnitude spectrogram and a phase spectrogram;
- splitting the magnitude spectrogram into K small fragments along the time dimension;
- sending each fragment in the K small fragments through a convolutional deep neural network to produce K small fragment outputs, wherein the convolutional deep neural network includes convolutional layers augmented with an attention layer to help the convolutional deep neural network find the most informative areas and reduce artifacts in each of the K small fragments, wherein the attention layer informs the convolutional deep neural network of important pixels;
- producing a source mask based on the K small fragment outputs, wherein the source mask corresponds to an audio source targeted to be isolated;
- multiplying the source mask with the magnitude spectrogram to create a new magnitude spectrogram corresponding to the source.
6 more elements
A non-transitory computer readable medium storing instructions to be executed by a processor, the instructions comprising:
- transforming an audio file into a complex spectrogram;
- decomposing the complex spectrogram into a magnitude spectrogram and a phase spectrogram;
- splitting the magnitude spectrogram into K small fragments along the time dimension;
- sending each fragment in the K small fragments through a convolutional deep neural network to produce K small fragment outputs, wherein the convolutional deep neural network includes convolutional layers augmented with an attention layer to help the convolutional deep neural network find the most informative areas and reduce artifacts in each of the K small fragments, wherein the attention layer informs the convolutional deep neural network of important pixels;
- producing a source mask based on the K small fragment outputs, wherein the source mask corresponds to an audio source targeted to be isolated;
- multiplying the source mask with the magnitude spectrogram to create a new magnitude spectrogram corresponding to the source.
3 more elements
Dependencies are inferred from the claim text.