# US 10,482,856 B2 · Automatic performance system, automatic performance method, and sign action learning method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,856 B2 - Document ID: US10482856B2 - Status: Granted - Kind: B2 - Application number: 16/256,103 - Filed: 2019-01-24 - Published: 2019-05-23 - Granted: 2019-11-19 - Earliest claimed: 2016-05-18 - Assignees: Yamaha Corporation - Applicants: Yamaha Corporation - Inventors: Akira Maezawa; Kazuhiko Yamamoto - Web page: https://patentgrep.com/patent/US10482856B2 ## CPC - G10H 1/0008: {Associated control or indicating means} - G10H 1/36: Accompaniment arrangements - G10H 1/361: {Recording/reproducing of accompaniment for use with an external source, e.g. karaoke systems} - G10H 2210/076: for extraction of timing, tempo; Beat detection - G10H 2210/091: for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance - G10H 2220/121: for graphical editing of a musical score, staff or tablature - G10H 2220/131: for abstract geometric visualisation of music, e.g. for interactive editing of musical parameters linked to abstract geometric figures - G10H 2220/201: for movement interpretation, i.e. capturing and recognizing a gesture or a specific kind of movement, e.g. to control a musical instrument - G10H 2220/455: Camera input, e.g. analyzing pictures from a video camera and using the analysis results as control data - G10H 2240/325: Synchronizing two or more audio tracks or files according to musical features or musical timings - G10H 2250/311: Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation - G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE ## Abstract An automatic performance system includes a sign detector configured to detect a sign action of a performer performing a musical piece, a performance analyzer configured to sequentially estimates a performance position in the musical piece by analyzing an acoustic signal representing performed sound in parallel with the performance, and a performance controller configured to control an automatic performance device to carry out an automatic performance of the musical piece so that the automatic performance is synchronized with the sign action detected by the sign detector and a progress of the performance position estimated by the performance analyzer. ## Claims ### Claim 1 An automatic performance system for a collaborative performance of a musical piece by at least one performer and an automatic performance device, the automatic performance system comprising: a controller, including at least one processor or circuit, configured to execute a plurality of tasks, including: a sign action detecting task that detects a sign action of the at least one performer performing the musical piece based on a received detection signal; a performance analyzing task that estimates a performance position in the musical piece based on a received acoustic signal representing a performed sound generated currently by the at least one performer; and a display control task that controls a display device to display an image corresponding to performance contents at the performance position in the musical piece estimated by the performance analyzing task. ### Claim 2 (depends on 1) The automatic performance system according to claim 1, wherein the displayed image represents a progress of the performance position in the musical piece based on the received acoustic signal of an automatic performance by the automatic performance device. ### Claim 3 (depends on 2) The automatic performance system according to claim 2, wherein a shape of the displayed image changes while a performed sound is being generated by the automatic performance device. ### Claim 4 (depends on 3) The automatic performance system according to claim 3, wherein the shape of the displayed image changes irregularly while the performed sound is being generated by the automatic performance device. ### Claim 5 (depends on 2) The automatic performance system according to claim 2, wherein the displayed image moves upwardly when a sound generation by the automatic performance device is started. ### Claim 6 (depends on 2) The automatic performance system according to claim 2, wherein the displayed image moves downwardly when a sound generation by the automatic performance device is stopped. ### Claim 7 An automatic performance method for an automatic performance system for a collaborate performance of a musical piece by at least one performer and an automatic performance device, the method comprising: a sign action detecting step of detecting a sign action of the at least one performer performing the musical piece based on a received detection signal; a performance analyzing step of estimating a performance position in the musical piece based on a received acoustic signal representing a performed sound generated currently by the at least one performer; and a display control step of controlling a display device to display an image corresponding to performance contents at the performance position in the musical piece estimated in the performance analyzing step. ### Claim 8 (depends on 7) The automatic performance method according to claim 7, wherein the displayed image represents a progress of the performance position in the musical piece based on the received acoustic signal of an automatic performance by the automatic performance device. ### Claim 9 (depends on 7) The automatic performance method according to claim 7, wherein the display control step changes a shape of the displayed image while a performed sound is being generated by the automatic performance device. ### Claim 10 (depends on 9) The automatic performance method according to claim 9, wherein the display control step irregularly changes the shape of the displayed image while the performed sound is being generated by the automatic performance device. ### Claim 11 (depends on 8) The automatic performance method according to claim 8, wherein the display control step moves the displayed image upwardly when a sound generation by the automatic performance device is started. ### Claim 12 (depends on 8) The automatic performance method according to claim 8, wherein the display control step moves the displayed image downwardly when a sound generation by the automatic performance device is stopped.