# US 9,857,934 B2 · Synchronized display and performance mapping of musical performances submitted from remote locations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,857,934 B2
- Document ID: US9857934B2
- Status: Granted
- Kind: B2
- Application number: 14/306,224
- Filed: 2014-06-16
- Published: 2015-02-12
- Granted: 2018-01-02
- Earliest claimed: 2013-06-16
- Assignees: Jammit, Inc.
- Applicants: Jammit, Inc.
- Inventors: Scott Humphrey; Frank Gryner
- Web page: https://patentgrep.com/patent/US9857934B2

## CPC

- G10H 1/368: {displaying animated or moving pictures synchronized with the music or audio part}
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G10H 1/0058: {Transmission between separate instruments or between individual components of a musical system (G10H1/0083 takes precedence)}
- H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46)
- G10H 2210/091: for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
- G10H 2220/106: using icons, e.g. selecting, moving or linking icons, on-screen symbols, screen regions or segments representing musical elements or parameters
- G10H 2240/171: Transmission of musical instrument data, control or status information; Transmission, remote access or control of music data for electrophonic musical instruments
- G10H 2240/175: for jam sessions or musical collaboration through a network, e.g. for composition, ensemble playing or repeating; Compensation of network or internet delays therefor
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Systems and methods are provided for assembling and displaying a visual ensemble of musical performances that were created and uploaded from one or more locations that are remote from a host of the network, a director or other administrator reviewing submissions for selection and assembly, or perhaps merely remote from one or more other submissions received over a computer network. The assembled performances include a plurality of submissions, the submissions including performances created and uploaded at one or more locations remote from the location of the director for the assembly and display over the computer network. Systems and methods are also included for mapping one performance against another performance qualitatively, quantitatively, in real-time, or some combination thereof, enabling a musician, or a reviewer of performances, in the assessment of one performance relative to another performance.

## Claims

### Claim 1

A system for creating an ensemble of a set of remotely-performed and remotely-uploaded musical performances through a computer network, the system comprising:
a processor operably connected to a memory on a non-transitory computer readable medium;
a receiving module operable for receiving audio data from a musical submission and a second musical submission on a non-transitory computer readable medium;
an integration engine embodied in a non-transitory computer readable medium, wherein the integration engine is operable to interface the timing of a musical submission with the timing a second musical submission using a timing reference track that includes a tempo map in the creation of an ensemble; and
a display module operable for displaying the ensemble; wherein the timing reference is selected from the group consisting of:
a click-track;
a drum track;
a select reference track of a select reference performance, , the select reference performance having a series of sound frequencies, each of the frequencies performed at a particular time in the select reference performance;
a residual audio track representing a multi-track digital audio file of a musical work having the select reference performance removed, the residual audio track having a series of sound frequencies, each of the frequencies performed at a particular time in the residual audio track; and,
a combination thereof.

### Claim 2 (depends on 1)

The system of claim 1, wherein the timing reference used by the integration engine is one track in each of a series of multi-track digital audio files available upon request by each of a plurality of musicians; each of the series containing a shared musical work, highlighting a select reference performance on a select reference track within the musical work, and including
(i) T reference performances within the musical work, wherein T=Σi; and, i=1, 2, 3, . . . , or . . . T;
(ii) a select reference track of the select reference performance, i, the select reference performance having a series of sound frequencies, each of the frequencies performed at a particular time in the select reference performance;
(iii) a residual audio track representing the multi-track digital audio file having the select reference performance removed; and,
(iv) instructions for execution of the timing reference by the processor.

### Claim 3 (depends on 1)

The system of claim 1, further comprising
an input device operable to receive video data on a non-transitory computer readable medium;
a video engine embodied in a non-transitory computer readable medium, wherein the video engine is operable to transform input video data to output video data as a video recording of a musical submission for display in a visual ensemble;
an output module embodied in a non-transitory computer readable medium, wherein the output module is operable to transmit the integrated audio data, along with the video data, to an output device, the output device operable to display the visual ensemble.

### Claim 4 (depends on 1)

The system of claim 1, further comprising a guidance module on a non-transitory computer readable medium and operable as a guidance system including a timing comparison tool to compare the musical submission to the select reference performance, the guidance system providing a feedback that includes a qualitative or quantitative measure of the timing comparison.

### Claim 5 (depends on 1)

The system of claim 1, further comprising a guidance module on a non-transitory computer readable medium and operable as a guidance system including a frequency comparison tool to compare the musical submission to the select reference performance, the guidance system providing a feedback that includes a qualitative or quantitative measure of the frequency comparison.

### Claim 6 (depends on 2)

The system of claim 2, further comprising a guidance module on a non-transitory computer readable medium and operable as a guidance system including a frequency comparison tool and a timing comparison tool to compare the musical submission to the select reference performance, the guidance system providing a feedback that includes a qualitative or quantitative measure of the frequency comparison and the timing comparison.

### Claim 7 (depends on 2)

The system of claim 2, further comprising a dynamic guidance module on a non-transitory computer readable medium and operable as a dynamic guidance system including a frequency comparison tool and a timing comparison tool to provide a continuous comparison of the musical submission with the select reference performance from the beginning of the performance to the end of the performance, or any section therein, the dynamic guidance system providing a qualitative or quantitative measure of the frequency comparison and the timing comparison.

### Claim 8 (depends on 2)

The system of claim 2, further comprising a dynamic guidance module on a non-transitory computer readable medium and operable as a dynamic guidance system, the guidance system including a frequency comparison tool and a timing comparison tool for each of the plurality of musicians to continuously compare their performance of the musical submission with the select reference performance while they perform, the dynamic guidance system providing a real-time feedback that includes a qualitative or quantitative measure of the frequency comparison and the timing comparison, the real-time feedback being no slower than 30 milliseconds from the occurrence of a respective transient or note performed in the musical submission and used for comparison to the select reference performance.