System and method for automated real-time feedback of a musical performance
AssigneeMagic Sheet Music Incorporated
InventorsKonrad Binienda, Kenrick Licorish
Life of this patent
Estimating the term…
Abstract
A music performance feedback tool is described herein that enables a musician to play freely and receive feedback in real-time. The tool tracks the musician as they progress through a piece and returns optimal feedback given an input sequence produced by the musician. The tool enables musicians to practice in a fashion that is most natural and pedagogically correct. The tool provides musicians with annotations indicating what notes in the score they failed to play, erroneously added notes, and the like. The tool further provides the musician with tempo and rhythmic feedback.
Claims20
A system, comprising:
- a processor coupled to a memory having computer-executable instructions that, when executed by the processor, configure the processor to:
- generate an input transcript from input of a musical performance;
- generate a reference transcript from a musical score associated with the input;
- determine an alignment between the input transcript and the reference transcript using a directed graph technique;
- and
- output feedback on the musical performance based on the alignment determined.
A method for providing real time feedback on a musical performance, comprising:
- acquiring input of the musical performance;
- generating an input transcript of the musical performance based on the input;
- comparing the input transcript to a reference transcript of a musical score associated with the musical performance to determine an alignment;
- generating annotations based on the alignment, wherein the annotations include corrective transformations to the musical performance;
- and
- displaying, in real time, the annotations on a rendered representations of the musical score as feedback to a performer.
A non-transitory, computer-readable storage medium having stored thereon computer-executable instructions that, when executed by a processor, configure the processor to:
- acquire digital input of the musical performance captured by an input device;
- process a sample of the digital input and provide processed digital input to neural network to generate active note probabilities;
- apply a threshold to the active note probabilities to determine actual active notes;
- group active notes occurring within a predetermined amount of time;
- append grouped active notes associated with the sample to a data structure representing an input transcript of the musical performance;
- generate a grid graph based on the input transcript and a reference transcript corresponding to a musical score associated with the musical performance;
- traverse the grid graph to identify one or more paths of alignment;
- evaluate the one or more paths of alignment based on one or more criteria;
- select an alignment from the one or more paths of alignment as a solution alignment;
- identify a series of edits, based on the solution alignment, to transform the input transcript into the reference transcript;
- generate annotations based on the series of edits;
- and
- output display data for displaying the annotations in connection with a visual representation of the musical score in real-time as feedback on the musical performance,
- wherein the display data is automatically updated to change the visual representation of the musical score during the performance to simulate a page turn event.
Dependencies are inferred from the claim text.