# US 2010/0145892 A1 · Search device and associated methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2010/0145892 A1
- Document ID: US20100145892A1
- Status: Published application
- Kind: A1
- Application number: 12/423,988
- Filed: 2009-04-15
- Published: 2010-06-10
- Earliest claimed: 2008-12-10
- Assignees: National Taiwan University
- Inventors: Yi-Hsuan Yang; Homer-H Chen
- Web page: https://patentgrep.com/patent/US20100145892A1

## CPC

- G06F 16/683: using metadata automatically derived from the content
- G06F 16/632: Query formulation
- G06F 16/639: {using playlists}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A search device and associated methods use music emotions to browse, organize and retrieve music collections. The search device comprises a processor and an interface. The processor uses machine learning techniques to determine music emotion according to music features and organizes music by emotions for browsing and retrieving music collections. The interface connects to the processor and allows a person to retrieve desired music from the processor. Methods associated with the search device comprise a processor initialization method, a method of loading new music into the search device and several methods of retrieving desired music from the search device.

## Claims

### Claim 1

A search device for browsing and retrieving music collections, comprising:
a processor organizing music by emotion; and
an interface connecting to the processor, allowing a person to input a description of emotion and retrieve the desired music based on the music emotion.

### Claim 2 (depends on 1)

The search device of claim 1, wherein the processor further comprises:
a database storing multiple music collections, music information and corresponding emotional values;
a feature extractor extracting features of music; and
an emotional predictor predicting music's emotion using machine learning techniques based on extracted features and giving each musical selection two emotional values comprising an arousal value (Ra) and a valence value (Rv).

### Claim 3 (depends on 1)

The search device of claim 1, wherein the interface further comprises a platform and wherein the platform is a two-dimensional coordinate system comprising:
multiple parallel longitudinal arousal sensors; and
multiple parallel valence sensors intersecting the longitudinal sensors and defining multiple emotional coordinates Ei (Ra, Rv).

### Claim 4 (depends on 2)

The search device of claim 2, wherein the features of music are extracted from audio spectrums, frequencies, harmonic signal distributions, power distribution, loudness, level, pitch, dissonance, timbral texture, rhythmic content and pitch content.

### Claim 5 (depends on 3)

The search device of claim 3, wherein the platform is a flat panel touch display.

### Claim 6

A processor initialization method for a search device using a processor organizing music by emotions for browsing and retrieving music collections and an interface allowing a person to input a description of emotion and retrieve the desired music based on the music emotion, the processor initialization method using machine learning techniques to build a data model for music emotion predictions and comprising:
sampling comprising choosing multiple clips of music from different fields and languages, converting the music clips to a uniform audio format;
subjective testing comprising surveying a group of people, recording their emotional assessment of the music clip into two basic values and averaging the basic values as a basis for the music emotion for each music clip;
feature extraction comprising extracting features of each music clip; and
setting data model comprising setting a data model for the processor, executing at least one regression algorithm to formulate the prediction of music emotion into a regression function.

### Claim 7 (depends on 6)

The processor initialization method of claim 6, wherein the act of feature extraction applies:
a conventional spectral contrast algorithm that extracts features through audio spectrums, frequencies and harmonic signal distributions of the clipped music;
a conventional Daubechies wavelets coefficient histogram (DWCH) algorithm that extracts features by calculating the average, variance or skewness power distribution of the music clip;
a conventional PsySound program that extracts features of loudness, level, pitch and dissonance from the music clip; and/or
a conventional Marsyas program that extracts features of timbral texture, rhythmic content and pitch content from the music clip.

### Claim 8 (depends on 6)

The processor initialization method of claim 6, wherein the at least one conventional regression algorithm in the act of setting a data model is selected from a multiple linear regression (MLR), a support vector regression (SVR) and an AdaBoost RT algorithm.

### Claim 9

A method of loading new music for a search device using a processor organizing music by emotions for browsing and retrieving music collections and an interface allowing a person to input a description of emotion and retrieve the desired music based on the music emotion, the method of loading new music predicting emotional values of new music collections, sensing the emotional coordinates of the desired music emotion, retrieving corresponding music and comprising:
extracting music features comprising extracting features of new music;
determining emotional values comprising predicting each new music emotional values; and
matching emotional values comprising selecting nearest emotional values to desired music emotion and retrieving corresponding music.

### Claim 10

A retrieval method for a search device using a processor organizing music by emotions for browsing and retrieving music collections and an interface allowing a person to input a description of emotion and retrieve the desired music based on the music emotion, the retrieval method being selected from the group consisting of: a query-by-emotion-point (QBEP) method; a query-by-emotion-trajectory (QBET) method; a query-by-artist-emotion (QBAE) method; and a query-by-emotion-region (QBER) method.

### Claim 11 (depends on 10)

The retrieval method of claim 10, wherein the QBEP method comprises:
emotional coordinate indication comprising indicating an emotional coordinate of desired music emotion; and
music collection retrieval comprising receiving the emotional coordinate and retrieving music with the nearest emotional coordinate.

### Claim 12 (depends on 10)

The retrieval method of claim 10, wherein the QBET method comprises:
emotional coordinate indication comprising sensing a trajectory of desired music emotion and quantizing emotional coordinates in the trajectory; and
music collection retrieval comprising retrieving corresponding music collections and generating a music playlist of the music collections.

### Claim 13 (depends on 10)

The retrieval method of claim 10, wherein the QBAE method is a combination of the QBEP method and a conventional artist-based retrieval method, retrieves corresponding music collections of a particular artist with corresponding music's emotions and comprises:
desired music emotion determination;
music collections retrieval matching the artist of the desired music emotion and labeling matched emotional coordinates; and
artist retrieval generating an artist list corresponding to the emotional coordinates.

### Claim 14 (depends on 13)

The retrieval method of claim 13, wherein the act of desired music emotion determination is selection of emotional coordinates.

### Claim 15 (depends on 13)

The retrieval method of claim 13, wherein the act of desired music emotion determination is selection of an artist.

### Claim 16 (depends on 10)

The retrieval method of claim 10, wherein the QBER method retrieves multiple pieces of music to generate a music playlist in a corresponding region and comprises:
emotional coordinate indication sensing a free region of a desired music emotion and quantizing the emotional coordinates in the region; and
music collection retrieval comprising retrieving corresponding music collections of the emotional coordinates and generating a music playlist of the music collections.
