# US 2018/0005614 A1 · Intelligent Crossfade With Separated Instrument Tracks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0005614 A1
- Document ID: US20180005614A1
- Status: Published application
- Kind: A1
- Application number: 15/198,499
- Filed: 2016-06-30
- Published: 2018-01-04
- Earliest claimed: 2016-06-30
- Assignees: Nokia Technologies Oy
- Applicants: Nokia Technologies Oy
- Inventors: Miikka Tapani Vilermo; Arto Juhani Lehtiniemi; Lasse Juhani Laaksonen; Mikko Tapio Tammi
- Web page: https://patentgrep.com/patent/US20180005614A1

## CPC

- G10H 1/0008: {Associated control or indicating means}
- G10H 1/46: Volume control
- H04H 60/04: Studio equipment; Interconnection of studios
- G10H 2210/036: of musical genre, i.e. analysing the style of musical pieces, usually for selection, filtering or classification
- G10H 2210/056: for extraction or identification of individual instrumental parts, e.g. melody, chords, bass; Identification or separation of instrumental parts by their characteristic voices or timbres
- G10H 2210/076: for extraction of timing, tempo; Beat detection
- G10H 2210/125: Medley, i.e. linking parts of different musical pieces in one single piece, e.g. sound collage, DJ mix
- G10H 2250/035: Crossfade, i.e. time domain amplitude envelope control of the transition between musical sounds or melodies, obtained for musical purposes, e.g. for ADSR tone generation, articulations, medley, remix
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- H04H: BROADCAST COMMUNICATION (multiplex communication H04J; pictorial communication aspects of broadcast systems H04N)

## Abstract

A method is provided including separating a first file into a first plurality of instrument tracks and a second file into a second plurality of instrument tracks, wherein each instrument track of each of the first plurality and second plurality corresponds to a type of instrument; selecting a first instrument track from the first plurality of instrument tracks and a second instrument track from the second plurality of instrument tracks based at least on the type of instrument corresponding to the first instrument track and the second instrument track; fading out other instrument tracks from the first plurality of instrument tracks; performing a crossfade between the first instrument track and the second instrument track; and fading in other instrument tracks from the second plurality of instrument tracks.

## Claims

### Claim 1

A method comprising:
separating a first file into a first plurality of instrument tracks and a second file into a second plurality of instrument tracks, wherein each instrument track of each of the first plurality and second plurality corresponds to a type of instrument;
selecting a first instrument track from the first plurality of instrument tracks and a second instrument track from the second plurality of instrument tracks based at least on a similarity of the first instrument track and the second instrument track;
fading out other instrument tracks from the first plurality of instrument tracks;
performing a crossfade between the first instrument track and the second instrument track; and
fading in other instrument tracks from the second plurality of instrument tracks.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining a dominant instrument in the first plurality of instrument tracks and a corresponding instrument track in the second plurality of instrument tracks comprising the dominant instrument, wherein the selecting comprises: selecting the dominant instrument track as the first instrument track and the corresponding instrument track as the second instrument track.

### Claim 3 (depends on 2)

The method of claim 2, wherein, if none of the instrument tracks in the second plurality of instrument tracks comprises the dominant instrument, the method further comprises at least one of:
determining a different dominant instrument in the first plurality of instrument tracks, wherein each of the selected first instrument track and the selected second instrument track comprises the different dominant instrument; and
determining a similar instrument in the second plurality of instrument tracks as the dominant instrument, wherein the selected first instrument track comprises the dominant instrument, and the selected second instrument track comprises the similar instrument.

### Claim 4 (depends on 1)

The method of claim 1, wherein the method further comprises:
creating a first group of instrument tracks by finding at least one further instrument track in the first plurality that is similar to the selected first instrument track;
creating a second group of instrument tracks by finding at least one further instrument track in the second plurality that is similar to the selected second instrument track; and
performing the crossfade between the first group of instrument tracks and the second group of instrument tracks.

### Claim 5 (depends on 4)

The method of claim 4, wherein finding similar instrument tracks is based on comparing at least one of: loudness, timbre, direction, and zero-crossing rate.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
determining a difference in tempo between the first instrument track and the second instrument track; and
adjusting, during the crossfade, the tempo of at least one of the first track and the second track based on the type of instrument.

### Claim 7 (depends on 1)

The method of claim 1, wherein at least one of:
the fading out comprises fading out each track in the first plurality of instrument tracks other than the selected first instrument track;
the fading in comprises fading in each track in the second plurality of instrument tracks other than the selected second instrument track.

### Claim 8 (depends on 1)

The method of claim 1, wherein, during at least a portion of the crossfade, at least one of:
one or more instrument tracks from the first plurality of instrument tracks that are different from the selected first instrument track are silenced; and
one or more instrument tracks from the second plurality of instrument tracks that are different from the selected second instrument track are silenced.

### Claim 9 (depends on 1)

The method of claim 1, wherein the separation is based on at least one of: MPEG Spatial Audio Object Coding (SAOC) and blind single sound source separation (BSS).

### Claim 10

An apparatus, comprising:
at least one processor; and
at least one memory including computer program code, the at least one memory and the computer program code configured, with the at least one processor, to cause the apparatus to perform at least the following:
separate a first file into a first plurality of instrument tracks and a second file into a second plurality of instrument tracks, wherein each instrument track of each of the first plurality and second plurality corresponds to a type of instrument;
select a first instrument track from the first plurality of instrument tracks and a second instrument track from the second plurality of instrument tracks based at least on a similarity of the first instrument track and the second instrument track;
fade out other instrument tracks from the first plurality of instrument tracks;
perform a crossfade between the first instrument track and the second instrument track; and
fade in other instrument tracks from the second plurality of instrument tracks.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to:
determine a dominant instrument in the first plurality of instrument tracks and a corresponding instrument track in the second plurality of instrument tracks comprising the dominant instrument, wherein the selection comprises: selection of the dominant instrument track as the first instrument track and the corresponding instrument track as the second instrument track.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein, if none of the instrument tracks in the second plurality of instrument tracks comprises the dominant instrument, the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to perform at least one of:
determine a different dominant instrument in the first plurality of instrument tracks, wherein each of the selected first instrument track and the selected second instrument track comprises the different dominant instrument; and
determine a similar instrument in the second plurality of instrument tracks as the dominant instrument, wherein the selected first instrument track comprises the dominant instrument, and the selected second instrument track comprises the similar instrument.

### Claim 13 (depends on 10)

The apparatus of claim 10, wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to perform at least the following:
create a first group of instrument tracks by finding at least one further instrument track in the first plurality that is similar to the selected first instrument track;
create a second group of instrument tracks by finding at least one further instrument track in the second plurality that is similar to the selected second instrument track; and
perform the crossfade between the first group of instrument tracks and the second group of instrument tracks.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein finding similar instrument tracks is based on comparing at least one of: loudness, timbre, direction, and zero-crossing rate.

### Claim 15 (depends on 10)

The apparatus of claim 10, wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to perform at least the following:
determine a difference in tempo between the first instrument track and the second instrument track; and
adjust, during the crossfade, the tempo of at least one of the first instrument track and the second instrument track.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein adjustment of the tempo comprises selecting a tempo manipulation algorithm based on the type of instrument.

### Claim 17 (depends on 10)

The apparatus of claim 10, wherein at least one of:
the fade out comprises fading out each instrument track in the first plurality of instrument tracks other than the selected first instrument track; and
the fade in comprises fading in each instrument track in the second plurality of instrument tracks other than the selected second instrument track.

### Claim 18 (depends on 10)

The apparatus of claim 10, wherein the separation is based on at least one of: MPEG Spatial Audio Object Coding (SAOC) and blind single sound source separation (BSS).

### Claim 19 (depends on 10)

The apparatus of claim 10, wherein the at least one memory and the computer program code are configured, with the at least one processor, to cause the apparatus to perform at least the following:
create a third file comprising the crossfade, and store the third file in the memory for audio playback.

### Claim 20

A computer program product comprising a non-transitory computer-readable storage medium having computer program code embodied thereon which when executed by an apparatus causes the apparatus to perform:
separating a first file into a first plurality of instrument tracks and a second file into a second plurality of instrument tracks, wherein each of the instruments track of each of the first plurality and the second plurality correspond to a type of instrument;
selecting a first instrument track from the first plurality of instrument tracks and a second instrument corresponding to the first instrument track and the second instrument track based at least on a similarity between the first instrument track and the second instrument track;
fading out other instrument tracks from the first plurality of instrument tracks;
performing a crossfade between the first instrument track and the second instrument track; and
fading in other instrument tracks from the second plurality of instrument tracks.
