# US 2023/0067175 A1 · Automatic music document displaying on performing music

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0067175 A1
- Document ID: US20230067175A1
- Status: Published application
- Kind: A1
- Application number: 17/897,546
- Filed: 2022-08-29
- Published: 2023-03-02
- Earliest claimed: 2021-09-01
- Assignees: Yousician Oy
- Applicants: Yousician Oy
- Inventors: Jarmo HIIPAKKA; Christoph Thür
- Web page: https://patentgrep.com/patent/US20230067175A1

## CPC

- G10H 1/0016: {Means for indicating which keys, frets or strings are to be actuated, e.g. using lights or leds}
- G10H 1/40: Rhythm
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/0483: Interaction with page-structured environments, e.g. book metaphor
- G06F 3/0485: Scrolling or panning
- G10G 1/00: Means for the representation of music
- G10H 2220/015: Musical staff, tablature or score displays, e.g. for score reading during a performance
- G10H 2220/096: using a touch screen
- G10H 2220/121: for graphical editing of a musical score, staff or tablature
- G10H 2240/131: Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10G: REPRESENTATION OF MUSIC; RECORDING MUSIC IN NOTATION FORM; ACCESSORIES FOR MUSIC OR MUSICAL INSTRUMENTS NOT OTHERWISE PROVIDED FOR, e.g. SUPPORTS (music stands A47B; non-musical aspects of musical toy instruments A63H5/00; metronomes G04F5/02; teaching music G09B15/00)

## Abstract

A user interface presents structural musical information in a score in a way where both the start and the end points of each jump in the score are visible simultaneously. Each jump is presented in a manner that allows the user to select during performance, which one of different alternatives to choose, when approaching a decision point like a repeat in the song.

## Claims

### Claim 1

A computer implemented method comprising
maintaining a music document indicating what should be performed in a piece of music; and repeatedly:
displaying on a display in a main view a first part of the music document when a user performs the piece of music;
identifying a source portion of the music document that is being performed by the user;
identifying in the musical document a plurality of potential jump targets including at least any positions that are structurally associated with the source portion;
displaying on the display one or more of the potential jump targets in respective one or more target views together with the main view;
identifying a destination portion of the musical document for performing after the source portion; and
updating the main view to display a second part of the musical document at the destination portion.

### Claim 2 (depends on 1)

The method of claim 1, further comprising in the identifying of the destination portion, receiving a user indication of a chosen jump target from the user or another person.

### Claim 3 (depends on 2)

The method of claim 2, further comprising receiving the user indication using one or more of any one of: a hard key; a soft key; a touch screen; a button.

### Claim 4 (depends on 2)

The method of claim 2, further comprising receiving the user indication using a gaze detection and positions of the main view and the target views on the display.

### Claim 5 (depends on 2)

The method of claim 2, further comprising receiving the user indication using an audio signal produced by the user.

### Claim 6 (depends on 5)

The method of claim 5, the identifying of the destination portion comprising detecting which part of the piece of music the user is performing.

### Claim 7 (depends on 5)

The method of claim 5, the identifying of the destination portion comprising recognizing what the user is performing by singing, using lyrics of the musical document.

### Claim 8 (depends on 5)

The method of claim 5, the identifying of the destination portion comprising using speech recognition and determining verbal indication of the destination portion.

### Claim 9 (depends on 5)

The method of claim 5, the identifying of the destination portion comprising using symbolic information of the music document, which symbolic information describes the piece of music.

### Claim 10 (depends on 9)

The method of claim 9, further comprising adding supplementary symbolic information to the music document allowing the user to indicate the destination portion by performing accordingly.

### Claim 11 (depends on 10)

The method of claim 10, wherein the symbolic information comprises an indication of notes or chords, and/or an indication of rhythm.

### Claim 12 (depends on 5)

The method of claim 5, wherein the source portion is the first part of the music document or a sub-part of the first part.

### Claim 13 (depends on 5)

The method of claim 5, further comprising
identifying in the musical document a plurality of structurally associated jump sources and jump targets; and
identifying one or more of the structurally associated jump sources in the source portion, and responsively identifying and displaying in respective target views one or more associated jump targets.

### Claim 14 (depends on 5)

The method of claim 5, further comprising in the identifying of the destination portion, receiving a user indication of the chosen jump target from another person.

### Claim 15 (depends on 5)

The method of claim 5, further comprising in the identifying of the destination portion, receiving a plurality of indications, and probabilistically determining the destination portion as the one that is most likely based on the plurality of indications.

### Claim 16 (depends on 15)

The method of claim 15, further comprising hierarchically prioritizing some of the plurality of indications.

### Claim 17 (depends on 16)

The method of claim 16, further comprising in the prioritizing, weighing more indications that are more reliable over indications that are less reliable.

### Claim 18 (depends on 5)

The method of claim 5, further comprising displaying the main view and the target views in an order corresponding to that of an original musical document.

### Claim 19 (depends on 5)

The method of claim 5, further comprising the music document comprising music sheets having lines and notes marked therein; the method further comprising adapting the music sheets by removing staves not needed for performing by the user.

### Claim 20 (depends on 1)

An apparatus comprising at least one memory and processor collectively configured to cause performing the method of claim 1.
