# US 12,183,363 B2 · Singing voice separation with deep u-net convolutional networks > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,363 B2 - Document ID: US12183363B2 - Status: Granted - Kind: B2 - Application number: 18/514,289 - Filed: 2023-11-20 - Published: 2024-05-16 - Granted: 2024-12-31 - Earliest claimed: 2018-08-06 - Assignees: Spotify AB - Applicants: Spotify AB - Inventors: Andreas Simon Thore Jansson; Angus William Sackfield; Ching Chuan Sung - Web page: https://patentgrep.com/patent/US12183363B2 ## CPC - G10H 1/0008: {Associated control or indicating means} - G10L 25/81: for discriminating voice from music - G06N 3/045: Combinations of networks - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 5/046: Forward inferencing; Production systems - G10H 1/361: {Recording/reproducing of accompaniment for use with an external source, e.g. karaoke systems} - G10L 15/16: using artificial neural networks - G10L 21/0272: Voice signal separating - G10L 21/10: Transforming into visible information - G06N 3/048: Activation functions - G10H 2210/056: for extraction or identification of individual instrumental parts, e.g. melody, chords, bass; Identification or separation of instrumental parts by their characteristic voices or timbres - G10H 2250/235: Fourier transform; Discrete Fourier Transform [DFT]; Fast Fourier Transform [FFT] - G10H 2250/311: Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation - G10L 25/30: using neural networks - G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A system, method and computer product for training a neural network system. The method comprises applying an audio signal to the neural network system, the audio signal including a vocal component and a non-vocal component. The method also comprises comparing an output of the neural network system to a target signal, and adjusting at least one parameter of the neural network system to reduce a result of the comparing, for training the neural network system to estimate one of the vocal component and the non-vocal component. In one example embodiment, the system comprises a U-Net architecture. After training, the system can estimate vocal or instrumental components of an audio signal, depending on which type of component the system is trained to estimate. ## Claims ### Claim 1 A method, comprising: receiving, by way of an input user interface of a media player, a selection of a play button; playing simultaneously, by way of an output user interface of the media player, and in response to the selection, two different versions of a musical song including a first version and a second version of the musical song, the first version including only an instrumental component of the musical song, and the second version including only a vocal component of the musical song; while the two different versions are being played simultaneously by way of the output user interface of the media player, receiving a volume adjustment by way of an adjustment of a volume control of the input user interface of the media player, the volume control being configured such that an adjustment of the volume control in a first direction increases a volume of a first of the two different versions being played by way of the output user interface of the media player and decreases a volume of a second of the two different versions being played by way of the output user interface of the media player, and such that an adjustment of the volume control in a second direction different from the first direction decreases the volume of the first of the two different versions being played by way of the output user interface of the media player and increases the volume of the second of the two different versions being played by way of the output user interface of the media player; and in response to receiving the volume adjustment, increasing a volume of one of the two different versions being played by way of the output user interface of the media player and decreasing a volume of the other of the two different versions being played by way of the output user interface of the media player. ### Claim 2 (depends on 1) The method of claim 1, wherein the two different versions of the musical song are two different versions of a same musical song. ### Claim 3 (depends on 2) The method of claim 2, wherein each of the two different versions is generated by isolating it from the same musical song. ### Claim 4 (depends on 3) The method of claim 3, wherein each of the two different versions is isolated from the same musical song using a U-net neural network. ### Claim 5 (depends on 1) The method of claim 1, wherein the increasing increases the volume of the one of the two different versions being played in inverse proportion to the volume of the other of the two different versions being played. ### Claim 6 (depends on 1) The method of claim 1, wherein the adjustment of the volume control includes a sliding of a slide bar; and wherein the first direction is opposite the second direction. ### Claim 7 (depends on 1) The method of claim 1, wherein the playing simultaneously plays in synchrony the two different versions of the musical song. ### Claim 8 A media player system, comprising: an input user interface an output user interface; a memory storing a program; and a processor coupled to the input user interface, the output user interface and the memory, the processor being controllable by the program to perform a method including: receiving, by way of the input user interface, a selection of a play button; playing simultaneously, by way of the output user interface and in response to the selection, two different versions of a musical song including a first version and a second version of the musical song, the first version including only an instrumental component of the musical song, and the second version including only a vocal component of the musical song; while the two different versions are being played simultaneously by way of the output user interface, receiving a volume adjustment by way of an adjustment of a volume control of the input user interface, the volume control being configured such that an adjustment of the volume control in a first direction increases a volume of a first of the two different versions being played by way of the output user interface and decreases a volume of a second of the two different versions being played by way of the output user interface, and such that an adjustment of the volume control in a second direction different from the first direction decreases the volume of the first of the two different versions being played by way of the output user interface and increases the volume of the second of the two different versions being played by way of the output user interface; and in response to receiving the volume adjustment, increasing a volume of one of the two different versions being played by way of the output user interface and decreasing a volume of the other of the two different versions being played by way of the output user interface. ### Claim 9 (depends on 8) The system of claim 8, wherein the two different versions of the musical song are two different versions of a same musical song. ### Claim 10 (depends on 9) The system of claim 9, wherein each of the two different versions is generated by isolating it from the same musical song. ### Claim 11 (depends on 10) The system of claim 10, wherein each of the two different versions is isolated from the same musical song using a U-net neural network. ### Claim 12 (depends on 8) The system of claim 8, wherein the increasing increases the volume of the one of the two different versions being played in inverse proportion to the volume of the other of the two different versions being played. ### Claim 13 (depends on 8) The system of claim 8, wherein the adjustment of the volume control includes a sliding of a slide bar; and wherein the first direction is opposite the second direction. ### Claim 14 (depends on 8) The system of claim 8, wherein the playing simultaneously plays in synchrony the two different versions of the musical song. ### Claim 15 A non-transitory computer-readable medium storing instructions which, when executed by a computer processor, causes the computer processor to perform a method, the method comprising: receiving, by way of an input user interface of a media player, a selection of a play button; playing simultaneously, by way of an output user interface of the media player, and in response to the selection, two different versions of a musical song including a first version and a second version of the musical song, the first version including only an instrumental component of the musical song, and the second version including only a vocal component of the musical song; while the two different versions are being played simultaneously by way of the output user interface of the media player, receiving a volume adjustment by way of an adjustment of a volume control of the input user interface of the media player, the volume control being configured such that an adjustment of the volume control in a first direction increases a volume of a first of the two different versions being played by way of the output user interface of the media player and decreases a volume of a second of the two different versions being played by way of the output user interface of the media player, and such that an adjustment of the volume control in a second direction different from the first direction decreases the volume of the first of the two different versions being played by way of the output user interface of the media player and increases the volume of the second of the two different versions being played by way of the output user interface of the media player; and in response to receiving the volume adjustment, increasing a volume of one of the two different versions being played by way of the output user interface of the media player and decreasing a volume of the other of the two different versions being played by way of the output user interface of the media player. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the two different versions of the musical song are two different versions of a same musical song. ### Claim 17 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein each of the two different versions is generated by isolating it from the same musical song; and wherein each of the two different versions is isolated from the same musical song using a U-net neural network. ### Claim 18 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the increasing increases the volume of the one of the two different versions being played in inverse proportion to the volume of the other of the two different versions being played. ### Claim 19 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the adjustment of the volume control includes a sliding of a slide bar; and wherein the first direction is opposite the second direction. ### Claim 20 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the playing simultaneously plays in synchrony the two different versions of the musical song.