# US 2021/0232965 A1 · Information processing apparatus, information processing method, and information processing program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0232965 A1
- Document ID: US20210232965A1
- Status: Published application
- Kind: A1
- Application number: 16/772,168
- Filed: 2019-10-10
- Published: 2021-07-29
- Earliest claimed: 2018-10-19
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: Taketo AKAMA
- Web page: https://patentgrep.com/patent/US20210232965A1

## CPC

- G06N 20/00: Machine learning
- G10H 1/0025: {Automatic or semi-automatic music composition, e.g. producing random music, applying rules from music theory or modifying a musical piece (automatically producing a series of tones G10H1/26)}
- G06F 16/285: {Clustering or classification}
- G06F 16/65: Clustering; Classification
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0475: Generative networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/094: Adversarial learning
- G06N 3/047: Probabilistic or stochastic networks
- G10H 2210/061: for extraction of musical phrases, isolation of musically relevant segments, e.g. musical thumbnail generation, or for temporal structure analysis of a musical piece, e.g. determination of the movement sequence of a musical work
- G10H 2210/131: Morphing, i.e. transformation of a musical piece into a new different one, e.g. remix
- G10H 2250/311: Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An information processing apparatus (100) according to the present disclosure includes an acquiring unit (132) that uses a trained encoder to acquire a feature value of each set of partial data forming first content; a calculating unit (133) that calculates a relative feature value that is a relative feature value between sets of partial data from the acquired feature values of the sets of partial data to calculate a relative feature value sequence indicating a feature of a structure of the first content; and a generating unit (134) that generates second content based on the relative feature value sequence of the first content and a feature value of optional data.

## Claims

### Claim 1

An information processing apparatus comprising:
an acquiring unit that uses a trained encoder to acquire a feature value of each set of partial data forming first content;
a calculating unit that calculates a relative feature value that is a relative feature value between sets of partial data from acquired feature values of the sets of partial data to calculate a relative feature value sequence indicating a feature of a structure of the first content; and
a generating unit that generates second content based on the relative feature value sequence of the first content and a feature value of optional data.

### Claim 2 (depends on 1)

The information processing apparatus according to claim 1, wherein the calculating unit executes addition, subtraction, multiplication, or division on a feature value of certain partial data of the first content and a feature value of each set of partial data other than the certain partial data to calculate the relative feature value so as to calculate the relative feature value sequence.

### Claim 3 (depends on 1)

The information processing apparatus according to claim 1, wherein the calculating unit calculates a relative feature value between a feature value of certain partial data of the first content and a feature value of each set of partial data that is other than the certain partial data and that has a causal relationship with the certain partial data so as to calculate the relative feature value sequence.

### Claim 4 (depends on 1)

The information processing apparatus according to claim 1, wherein the generating unit inputs a feature value sequence calculated from a relative feature value sequence of the first content and a feature value of optional data into a trained decoder to generate the second content.

### Claim 5 (depends on 1)

The information processing apparatus according to claim 1, wherein the generating unit generates an optional song as the second content based on a relative feature value sequence of the first content that is a song.

### Claim 6 (depends on 5)

The information processing apparatus according to claim 5, wherein
the acquiring unit inputs, as data indicating a sound included in the partial data and in the optional data, a symbol string indicating a pitch, a sound length, and a rest to the trained encoder to acquire feature values corresponding to the partial data and the optional data, and
the generating unit inputs the relative feature value sequence obtained based on acquired feature values of the optional data and the partial data to the trained decoder to generate the second content.

### Claim 7 (depends on 1)

The information processing apparatus according to claim 1, wherein
the acquiring unit acquires a feature value of each set of partial data forming the first content that is text data, and
the generating unit generates the second content that is text data based on a relative feature value sequence of the first content obtained based on the acquired feature value of each set of partial data and the feature value of the optional data that is text data.

### Claim 8 (depends on 1)

The information processing apparatus according to claim 1, wherein
the acquiring unit acquires a feature value of each set of partial data forming the first content that is moving image content, and
the generating unit generates the second content that is moving image content based on a relative feature value sequence of the first content obtained based on the acquired feature value of each set of partial data and the feature value of the optional data that is moving image or image content.

### Claim 9 (depends on 1)

The information processing apparatus according to claim 1, wherein the acquiring unit detects a break of a bar of the first content that is a song and acquires a feature value of each detected bar.

### Claim 10 (depends on 1)

The information processing apparatus according to claim 1, wherein the acquiring unit detects a rest that is more than a length of a predetermined threshold in the first content that is a song, divides the song into partial data based on the detected rest, and acquires a feature value of each set of divided partial data.

### Claim 11

An information processing method causing a computer to:
use a trained encoder to acquire a feature value of each set of partial data forming first content;
calculate a relative feature value that is a relative feature value between sets of partial data from acquired feature values of the sets of partial data to calculate a relative feature value sequence indicating a feature of a structure of the first content; and
generate second content based on the relative feature value sequence of the first content and a feature value of optional data.

### Claim 12

An information processing program causing a computer to function as:
an acquiring unit that uses a trained encoder to acquire a feature value of each set of partial data forming first content;
a calculating unit that calculates a relative feature value that is a relative feature value between sets of partial data from acquired feature values of the sets of partial data to calculate a relative feature value sequence indicating a feature of a structure of the first content; and
a generating unit that generates second content based on the relative feature value sequence of the first content and a feature value of optional data.
