Computer vision and mapping for audio

A system comprising:

• a wearable device comprising a processor, a memory, a camera, a loudspeaker, and a display;
• programming in the memory, wherein execution of the programming by the processor configures the wearable device to perform functions, including functions to:
• store in the memory a plurality of preset token color values, each associated with a set of sound samples;
• capture with the camera a color image of a note matrix comprising one or more fields defined by rows and columns, wherein each row is associated with a musical note and each column is associated with a beat duration;
• detect one or more tokens in the captured color image, wherein the one or more tokens comprises at least a first token characterized by a first token color value wherein the first token is detected at a first token position relative to the fields of the note matrix, and a subsequent token characterized by a subsequent token color value wherein the subsequent token is detected at a subsequent token position relative to the fields of the note matrix;
• retrieve from the memory a first set of sound samples based on the first token color value and a subsequent set of sound samples based on the subsequent token color value;
• select a first playable sound from the first set of sound samples based on the first token position and a subsequent playable sound from the subsequent set of sound samples based on the subsequent token position;
• and
• play through the loudspeaker a musical composition comprising the first playable sound and the subsequent playable sound in a sequence according to each column, in order, of the note matrix.

A non-transitory computer-readable medium storing program code which, when executed, is operative to cause an electronic processor to perform the steps of:

• initiating a computer vision and mapping application on a wearable device comprising a processor, a memory, a camera, a loudspeaker, and a display;
• storing in the memory a plurality of preset token color values, each associated with a set of sound samples;
• capturing with the camera a color image of a note matrix comprising one or more fields defined by rows and columns, wherein each row is associated with a musical note and each column is associated with a beat duration;
• detecting with the application one or more tokens in the captured color image, wherein the one or more tokens comprises at least a first token characterized by a first token color value wherein the first token is detected at a first token position relative to the fields of the note matrix and a subsequent token characterized by a subsequent token color value wherein the subsequent token is detected at a subsequent token position relative to the fields of the note matrix;
• retrieving from the memory a first set of sound samples based on the first token color value and a subsequent set of sound samples based on the subsequent token color value;
• selecting a first playable sound from the first set of sound samples based on the first token position and a subsequent playable sound from the subsequent set of sound samples based on the subsequent token position;
• and
• playing through the loudspeaker a musical composition comprising the first playable sound and the subsequent playable sound in a sequence according to each column, in order, of the note matrix.