# US 2020/0394989 A1 · Information processing method and apparatus for processing performance of musical piece

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394989 A1
- Document ID: US20200394989A1
- Status: Published application
- Kind: A1
- Application number: 17/005,924
- Filed: 2020-08-28
- Published: 2020-12-17
- Earliest claimed: 2018-03-01
- Assignees: Yamaha Corporation
- Applicants: Yamaha Corporation
- Inventors: Akira MAEZAWA
- Web page: https://patentgrep.com/patent/US20200394989A1

## CPC

- G10H 1/0008: {Associated control or indicating means}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G10G 1/00: Means for the representation of music
- G10H 1/0066: {using a MIDI interface}
- G10H 1/46: Volume control
- G10H 2210/051: for extraction or detection of onsets of musical sounds or notes, i.e. note attack timings
- G10H 2210/091: for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
- G10H 2210/165: Humanizing effects, i.e. causing a performance to sound less machine-like, e.g. by slightly randomising pitch or tempo
- G10H 2250/311: Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10G: REPRESENTATION OF MUSIC; RECORDING MUSIC IN NOTATION FORM; ACCESSORIES FOR MUSIC OR MUSICAL INSTRUMENTS NOT OTHERWISE PROVIDED FOR, e.g. SUPPORTS (music stands A47B; non-musical aspects of musical toy instruments A63H5/00; metronomes G04F5/02; teaching music G09B15/00)

## Abstract

Provided is an information processing apparatus that generates various kinds of time series data according to a performance tendency of a user. The information processing apparatus includes an index specifying unit 22 that specifies performance tendency information that indicates a performance tendency of a performance of a musical piece by a user by inputting observational performance data X representing the performance to a learned model La and an information processing unit 23 that generates time series data Z regarding the musical piece according to the performance tendency information.

## Claims

### Claim 1

An information processing method comprising:
generating performance tendency information indicating a performance tendency of a performance of a musical piece by a user from observational performance data representing the performance input to a learned model; and
generating time series data of the musical piece according to the generated performance tendency information.

### Claim 2 (depends on 1)

The information processing method according to claim 1, wherein:
the performance tendency information includes an index value indicating a probability of the performance corresponding to a performance tendency, from among a plurality of mutually different performance tendencies, and
the generating of the time series data generates the time series data from a plurality of sets of basic data respectively corresponding to the plurality of mutually different performance tendencies, according to the respective index values of the plurality of mutually different performance tendencies.

### Claim 3 (depends on 1)

The information processing method according to claim 1, wherein:
the generating of the performance tendency information generates the performance tendency information for each of a plurality of analysis periods on a time axis, using the observational performance data within the respective analysis period, and
the generating of the time series data generates, for each of the plurality of analysis periods, a portion of the time series data within the respective analysis period, according to the performance tendency information for the respective analysis period.

### Claim 4 (depends on 2)

The information processing method according to claim 2, wherein:
the generating of the performance tendency information generates the index values using the observational performance data for a plurality of analysis periods on a time axis, and
the generating of the time series data generates, for each of the plurality of analysis periods, portions of the plurality of sets of basic data within the respective analysis period, according to the respective index values of the performance tendency for the respective analysis period.

### Claim 5 (depends on 1)

The information processing method according to claim 1, wherein the time series data represents a time series of notes.

### Claim 6 (depends on 2)

The information processing method according to claim 2, wherein the time series data represents a time series of notes.

### Claim 7 (depends on 6)

The information processing method according to claim 6, wherein the generating of the time series data generates the time series data by compositing notes corresponding to each other in the plurality of sets of basic data, according to the respective index values of the plurality of mutually different performance tendencies.

### Claim 8 (depends on 7)

The information processing method according to claim 7, wherein:
the basic data of each of the plurality of mutually different performance tendencies represents time series of notes including common notes corresponding to each other in the plurality of sets of basic data and individual notes that differ for each set of basic data, and
the generating of the time series data generates the time series data by:
compositing the common notes corresponding to each other in the plurality of sets of basic data, according to the respective index values of the plurality of mutually different performance tendencies; and
selecting or not selecting the individual notes represented by the basic data corresponding to each performance tendency, according to the index value of the respective performance tendency.

### Claim 9 (depends on 1)

The information processing method according to claim 1, wherein the generating of the performance tendency information generates the performance tendency information by furnishing reference performance data representing the performance of the musical piece and the observational performance data to the learned model, the reference performance data being generated from musical piece data specifying a time series of notes constituting the musical piece.

### Claim 10 (depends on 1)

The information processing method according to claim 1, wherein:
the observational performance data includes sound volume data including a plurality of elements corresponding to mutually different sound volumes for each of a plurality of notes, and
an element, from among the plurality of elements in the sound volume data of each note, corresponding to a sound volume of the note played by the user is set to a first value, while remaining elements are set to a second value that differs from the first value.

### Claim 11

An information processing apparatus comprising:
a memory storing instructions; and
a processor that implements the instructions to execute a plurality of tasks, including:
a first generating task that generates performance tendency information indicating a performance tendency of a performance of a musical piece by a user from observational performance data representing the performance input to a learned model; and
a second generating task that generates time series data of the musical piece according to the generated performance tendency information.

### Claim 12 (depends on 11)

The information processing apparatus according to claim 11, wherein:
the performance tendency information includes an index value indicating a probability of the performance corresponding to a performance tendency, from among a plurality of mutually different performance tendencies, and
the second generating task generates the time series data from a plurality of sets of basic data respectively corresponding to the plurality of mutually different performance tendencies, according to the respective index values of the plurality of mutually different performance tendencies.

### Claim 13 (depends on 12)

The information processing apparatus according to claim 12, wherein:
the first generating task generates the index values using the observational performance data for a plurality of analysis periods on a time axis, and
the second generating task generates, for each of the plurality of analysis periods, portions of the plurality of sets of basic data within the respective analysis period, according to the respective index values of the performance tendency for the respective analysis period.

### Claim 14 (depends on 11)

The information processing apparatus according to claim 11, wherein the time series data represents a time series of notes.

### Claim 15 (depends on 12)

The information processing apparatus according to claim 12, wherein the time series data represents a time series of notes.

### Claim 16 (depends on 15)

The information processing apparatus according to claim 15, wherein the second generating task generates the time series data by compositing notes corresponding to each other in the plurality of sets of basic data, according to the respective index values of the plurality of mutually different performance tendencies.

### Claim 17 (depends on 16)

The information processing apparatus according to claim 16, wherein:
the basic data of each of the plurality of mutually different performance tendencies represents time series of notes including common notes corresponding to each other in the plurality of sets of basic data and individual notes that differ for each set of basic data, and
the second generating task generates the time series data by:
compositing the common notes corresponding to each other in the plurality of sets of basic data, according to the index values of the respective performance tendencies; and
selecting or not selecting the individual notes represented by the basic data corresponding to each performance tendency, according to the index value of the respective performance tendency.

### Claim 18 (depends on 11)

The information processing apparatus according to claim 11, wherein the first generating task generates the performance tendency information by furnishing reference performance data representing the performance of the musical piece and the observational performance data to the learned model, the reference performance data being generated from musical piece data that specifies a time series of notes constituting the musical piece.

### Claim 19 (depends on 11)

The information processing apparatus according to claim 11, wherein:
the observational performance data includes sound volume data including a plurality of elements corresponding to mutually different sound volumes for each of a plurality of notes, and
an element, from among the plurality of elements in the sound volume data of each note, corresponding to a sound volume of the note played by the user is set to a first value, while remaining elements are set to a second value that differs from the first value.

### Claim 20

A non-transitory computer readable medium storing a program executable by a computer to execute a method comprising:
generating performance tendency information indicating a performance tendency of a performance of a musical piece by a user from observational performance data representing the performance input to a learned model; and
generating time series data of the musical piece according to the performance tendency information.
