# US 2021/0279588 A1 · Automatic isolation of multiple instruments from musical mixtures

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279588 A1
- Document ID: US20210279588A1
- Status: Published application
- Kind: A1
- Application number: 17/205,296
- Filed: 2021-03-18
- Published: 2021-09-09
- Earliest claimed: 2018-08-06
- Assignees: Spotify AB
- Applicants: Spotify AB
- Inventors: Andreas Simon Thore Jansson; Angus William Sackfield; Ching Chuan Sung; Rachel M. Bittner
- Web page: https://patentgrep.com/patent/US20210279588A1

## CPC

- G10H 1/361: {Recording/reproducing of accompaniment for use with an external source, e.g. karaoke systems}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G10H 1/0008: {Associated control or indicating means}
- G10L 21/0272: Voice signal separating
- G10L 21/028: using properties of sound source
- G10H 2210/056: for extraction or identification of individual instrumental parts, e.g. melody, chords, bass; Identification or separation of instrumental parts by their characteristic voices or timbres
- G10H 2250/311: Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A system, method and computer product for training a neural network system. The method comprises inputting an audio signal to the system to generate plural outputs f(X, Θ). The audio signal includes one or more of vocal content and/or musical instrument content, and each output f(X, Θ) corresponds to a respective one of the different content types. The method also comprises comparing individual outputs f(X, Θ) of the neural network system to corresponding target signals. For each compared output f(X, Θ), at least one parameter of the system is adjusted to reduce a result of the comparing performed for the output f(X, Θ), to train the system to estimate the different content types. In one example embodiment, the system comprises a U-Net architecture. After training, the system can estimate various different types of vocal and/or instrument components of an audio signal, depending on which type of component(s) the system is trained to estimate.

## Claims

### Claim 1

A method for estimating a component of a provided audio signal, comprising:
converting the provided audio signal to an image;
inputting the image to a U-Net trained to estimate different types of content, the different types of content including one or more of vocal content and musical instrument content, wherein, in response to the input image, the U-Net outputs signals, each representing a corresponding one of the different types of content; and
converting each of the signals output by the U-Net to an audio signal.

### Claim at position 2

-20. (canceled)
