# US 11,027,204 B2 · Instrument game system and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,027,204 B2
- Document ID: US11027204B2
- Status: Granted
- Kind: B2
- Application number: 16/276,728
- Filed: 2019-02-15
- Published: 2019-07-11
- Granted: 2021-06-08
- Earliest claimed: 2007-02-20
- Assignees: Ubisoft Entertainment
- Applicants: Ubisoft Entertainment
- Inventors: Paul Cross; Nicholas Bonardi; Naomi Higo
- Web page: https://patentgrep.com/patent/US11027204B2

## CPC

- A63F 13/814: Musical performances, e.g. by evaluating the player's ability to follow a notation
- A63F 13/215: comprising means for detecting acoustic signals, e.g. using a microphone
- A63F 13/67: adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
- G10H 1/368: {displaying animated or moving pictures synchronized with the music or audio part}
- G10H 1/38: Chord
- A63F 13/245: specially adapted to a particular type of game, e.g. steering wheels
- A63F 13/44: involving timing of operations, e.g. performing an action within a time slot
- A63F 13/46: Computing the game score
- A63F 13/533: for prompting the player, e.g. by displaying a game menu
- A63F 2300/1062: being specially adapted to a type of game, e.g. steering wheel
- A63F 2300/6027: using adaptive systems learning from user actions, e.g. for skill level adjustment
- A63F 2300/61: Score computation
- A63F 2300/638: according to the timing of operation or a time limit
- A63F 2300/8047: Music games
- G10H 2210/091: for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
- G10H 2220/015: Musical staff, tablature or score displays, e.g. for score reading during a performance
- G10H 2220/151: Musical difficulty level setting or selection
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE

## Abstract

A game system and method that uses an instrument as an input encourages a user to play along with the game's soundtrack on an instrument (e.g. guitar, bass, etc.). The game cues the player to play notes and/or chords on the instrument at an appropriate time and then data is collected from the instrument via a connection between the instrument and the apparatus running the game. The game then scores the user based on note/chord and timing information it receives.

## Claims

### Claim 1

A system for instructing a user on playing a musical instrument, comprising:
a sound input for interfacing with the musical instrument configured to receive analog sound data from the musical instrument and using a processing unit, convert the sound data into performance notes, each performance note corresponding to one or more discrete musical notes;
one or more non-transitory computer-readable media storing:
a digital representation of a song, said song comprising a plurality of phrases, each phrase of the plurality of phrases comprising a plurality of arrangement notes;
computer-executable instructions that, when executed by a processor, perform a method of instructing the user on playing the musical instrument, the method comprising the steps of:
selecting, for a first phrase of the song, a first subset of the plurality of arrangement notes corresponding to the first phrase as cued notes for the first phrase;
presenting, to the user, a visual representation of the cued notes for the first phrase;
receiving, via the sound input, performance notes for the first phrase performed by the user in response to the visual representation of the cued notes for the first phrase,
updating an estimation of the user's skill level based on a comparison of the performance notes for the first phrase and the cued notes for the first phrase;
selecting, for a second phrase of the song, a second subset of the plurality of arrangement notes corresponding to the second phrase as cued notes for the second phrase;
wherein the cued notes for the second phrase are selected based on the updated estimation of the user's skill level; and
presenting, to the user, a visual representation of the cued notes for the second phrase.

### Claim 2 (depends on 1)

The system of claim 1, wherein the sound input for interfacing with the musical instrument takes the form of an amplifier cord for the musical instrument and comprises a USB output for transmitting the series of performance notes to the processor.

### Claim 3 (depends on 1)

The system of claim 1, wherein the sound input for interfacing with the musical instrument comprises a microphone for receiving the sound data and a network interface for transmitting the performance notes to the processor.

### Claim 4 (depends on 1)

The system of claim 1, wherein the sound input comprises a microphone for receiving a vocal performance.

### Claim 5 (depends on 1)

The system of claim 1, wherein the musical instrument is a piano keyboard.

### Claim 6 (depends on 1)

The system of claim 1, wherein the musical instrument is a stringed instrument.

### Claim 7 (depends on 1)

The system of claim 1, wherein the estimation of user's skill level comprises estimations of the user's skill at a plurality of techniques.

### Claim 8 (depends on 7)

The system of claim 7, wherein the plurality of techniques comprises single note techniques, chord techniques, and rhythm techniques.

### Claim 9 (depends on 1)

The system of claim 1, wherein the estimation of the user's skill level persists between songs.

### Claim 10 (depends on 1)

The system of claim 1, wherein the computer-readable media stores further computer-readable instructions operable to receive a digital representation of an additional song via a network interface.

### Claim 11

A method of instructing a user on playing a musical instrument, the method comprising the steps of:
selecting, for a first phrase of a song comprising a plurality of phrases, each phrase of the plurality of phrases comprising a plurality of arrangement notes, a first subset of the plurality of arrangement notes corresponding to the first phrase as cued notes for the first phrase;
presenting, to the user, a visual representation of the cued notes for the first phrase;
receiving, via a sound input for interfacing with the musical instrument, performance notes for the first phrase performed by the user in response to the visual representation of the cued notes for the first phrase,
wherein the sound input is configured to receive analog sound data from the musical instrument and, using a processing unit, convert the sound data into performance notes, each performance note corresponding to one or more discrete musical notes;
updating an estimation of the user's skill level based on a comparison of the performance notes for the first phrase and the cued notes for the first phrase;
selecting, for a second phrase of the song, a second subset of the plurality of arrangement notes corresponding to the second phrase as cued notes for the second phrase;
wherein the cued notes for the second phrase are selected based on the updated estimation of the user's skill level; and
presenting, to the user, a visual representation of the cued notes for the second phrase.

### Claim 12 (depends on 11)

The method of claim 11, wherein the sound input for interfacing with the musical instrument takes the form of an amplifier cord for the musical instrument and comprises a USB output for transmitting the series of performance notes to the processor.

### Claim 13 (depends on 11)

The method of claim 11, wherein the sound input for interfacing with the musical instrument comprises a microphone for receiving the sound data and a network interface for transmitting the performance notes to the processor.

### Claim 14 (depends on 11)

The method of claim 11, wherein the sound input comprises a microphone for receiving a vocal performance.

### Claim 15 (depends on 11)

The method of claim 11, wherein the musical instrument is a piano keyboard.

### Claim 16 (depends on 11)

The method of claim 11, wherein the musical instrument is a stringed instrument.

### Claim 17 (depends on 11)

The method of claim 11, wherein the estimation of user's skill level comprises estimations of the user's skill at a plurality of techniques.

### Claim 18 (depends on 17)

The method of claim 17, wherein the plurality of techniques comprises single note techniques, chord techniques, and rhythm techniques.

### Claim 19 (depends on 11)

The method of claim 11, wherein the estimation of the user's skill level persists between songs.

### Claim 20 (depends on 11)

The method of claim 11, further comprising the step of receiving the digital representation of the song via a network interface.