Neural networks for accompaniment extraction from songs

A system comprising:

• one or more computer processors;
• and
• one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the system to perform operations comprising:
• processing a first output image, generated with a first convolutional neural network (CNN), using a second CNN to generate a clean loss, the clean loss indicating an amount of the first output image that is not attributed to a first accompaniment portion;
• training the first CNN based on the clean loss;
• receiving an input song comprising a vocal portion and a second accompaniment portion;
• transforming the input song to an input image;
• processing the input image using the first CNN to generate a second output image;
• transforming the second output image to an output accompaniment, the output accompaniment comprising the second accompaniment portion of the input song;
• storing first feature values determined in processing the first output image by the second CNN;
• processing a modified ground truth output song with the second CNN;
• storing second feature values determined in processing the modified ground truth output song by the second CNN;
• determining a feature loss based on differences between the second feature values and the first feature values;
• and
• training the second CNN further based on the feature loss.

A non-transitory computer-readable storage medium including instructions that, when processed by a computer, configure the computer to perform operations comprising:

• processing a first output image, generated with a first convolutional neural network (CNN), using a second CNN to generate a clean loss, the clean loss indicating an amount of the first output image that is not attributed to a first accompaniment portion;
• training the first CNN based on the clean loss;
• receiving an input song comprising a vocal portion and a second accompaniment portion;
• transforming the input song to an input image;
• processing the input image using the first CNN to generate a second output image;
• transforming the second output image to an output accompaniment, the output accompaniment comprising the second accompaniment portion of the input song;
• storing first feature values determined in processing the first output image by the second CNN;
• processing a modified ground truth output song with the second CNN;
• storing second feature values determined in processing the modified ground truth output song by the second CNN;
• determining a feature loss based on differences between the second feature values and the first feature values;
• and
• training the second CNN further based on the feature loss.