Music visualization by translating music cues to visual cues

A computer-implemented method of presenting a visualization of a piece of music on a visual display, the method comprising:

• (a) providing a translation system, wherein the translation system comprises:
• a. a set of psychoacoustic cues, each psychoacoustic cue selected from a set of psychoacoustic cues, each psychoacoustic cue representing a distinct characteristic of the piece of music specific to music perception,
• b. an assignment of each selected psychoacoustic cue, or a rule-based combination of psychoacoustic cues, to a visual cue to provide a one-to-one correspondence between each selected psychoacoustic cue, or rule-based combination of psychoacoustic cues, and each visual cue, the assignment including rules for the spatial arrangement of visual cues on the visual display, wherein the assignment specifically aids in music perception, and wherein the assignment accounts for complexity, structure, and tempo of the piece of music and size and resolution of the visual display,
• c. where each visual cue specifically and individually represents a psychoacoustic cue and is different from a visual inference of that psychoacoustic cue based only on visual depiction of the basic audio cues of the notes involved in the psychoacoustic cue, those
• basic audio cues comprising pitches, times of onset and duration, amplitudes over time, and any note-specific embellishments, and
• d. where the translation system provides the ability to allocate visual cues associated with different representations of the music to different parts of the visual display, those representations including musical movements, phrases, chords, and notes, those representations can appear and disappear in the course of a musical piece, those parts of the visual display including spatial sections and layers;
• (b) extracting the selected psychoacoustic cues from the piece of music and translating, as a function of the translation system, the extracted psychoacoustic cues to corresponding visual cues and their spatial arrangement;
• and
• (c) presenting the visualization of the piece of music on the visual display by causing display of the visual cues on the visual display as the piece of music is being played, so that one or more persons sees the corresponding visual cues time synchronized with the piece of music as they hear the piece of music.

A computer-implemented method of presenting a visualization of a piece of music on a visual display, the method comprising:

• (a) providing a translation system, wherein the translation system comprises:
• a. a set of psychoacoustic cues, each psychoacoustic cue selected from a set of psychoacoustic cues, each psychoacoustic cue representing a distinct characteristic of the piece of music specific to music perception,
• b. an assignment of each selected psychoacoustic cue, or a rule-based combination of psychoacoustic cues, to a visual cue to provide a one-to-one correspondence between each selected psychoacoustic cue, or rule-based combination of psychoacoustic cues, and each visual cue, the assignment including rules for the spatial arrangement of visual cues on the visual display, wherein the assignment specifically aids in music perception, and wherein the assignment accounts for complexity, structure, and tempo of the piece of music and size and resolution of the visual display,
• c. where each visual cue specifically and individually represents a psychoacoustic cue and is different from a visual inference of that psychoacoustic cue based only on visual depiction of the basic audio cues of the notes involved in the psychoacoustic cue, those basic audio cues comprising pitches, times of onset and duration, amplitudes over time, and any note-specific embellishments, and
• d. where the translation system provides the ability to allocate visual cues associated with different source components of the music to different parts of the visual display, those source components including musical parts such as melody, harmonies, and rhythm, and different voices, instruments, and musicians, those parts of the visual display including spatial sections and layers;
• (b) extracting the selected psychoacoustic cues from the piece of music and translating, as a function of the translation system, the extracted psychoacoustic cues to corresponding visual cues and their spatial arrangement;
• and
• (c) presenting the visualization of the piece of music on the visual display by causing display of the visual cues on the visual display as the piece of music is being played, so that one or more persons sees the corresponding visual cues time synchronized with the piece of music as they hear the piece of music.

A computer-implemented method of presenting a visualization of a piece of music on a visual display, the method comprising:

• (a) providing a translation system, wherein the translation system comprises:
• a. a set of psychoacoustic cues, each psychoacoustic cue selected from a set of psychoacoustic cues, each psychoacoustic cue representing a distinct characteristic of the piece of music specific to music perception,
• b. an assignment of each selected psychoacoustic cue, or a rule-based combination of psychoacoustic cues, to a visual cue to provide a one-to-one correspondence between each selected psychoacoustic cue, or rule-based combination of psychoacoustic cues, and each visual cue, the assignment including rules for the spatial arrangement of visual cues on the visual display, wherein the assignment specifically aids in music perception, and wherein the assignment accounts for complexity, structure, and tempo of the piece of music and size and resolution of the visual display, and
• c. where each visual cue specifically and individually represents a psychoacoustic cue and is different from a visual inference of that psychoacoustic cue based only on visual depiction of the basic audio cues of the notes involved in the psychoacoustic cue, those basic audio cues comprising pitches, times of onset and duration, amplitudes over time, and any note-specific embellishments;
• (b) extracting the selected psychoacoustic cues from the piece of music and translating, as a function of the translation system, the extracted psychoacoustic cues to corresponding visual cues and their spatial arrangement;
• (c) combining the results of operation (b) above with, where available, one or more videos of the music performance or music videos associated with that performance, both of those video types representing visual cues and representations of the music to be allocated to parts of the visual display including spatial sections and layers;
• and
• (d) presenting the visualization of the piece of music on the visual display by causing display of the visual cues on the visual display as the piece of music is being played, so that one or more persons sees the corresponding visual cues time synchronized with the piece of music as they hear the piece of music.