Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting

Abstract

A system and method for real-time controlling of signal processors, synthesizers, musical instruments, MIDI processors, lighting, video, and special effects in performance, recording, and composition environments using images derived from tactile sensors, pressure sensor arrays, optical transducer arrays, chemical sensor arrays, body sensor arrays, and numerical computation. The invention provides for pressure sensor arrays and body sensor arrays as tactile control interfaces, for video cameras and light sensor arrays as optical transducers, for chemical sensor arrays, and for other numerical image generation from computer processing or numeric simulation. Tactile transducers may be put on instrument keys of conventional instruments, be attached to existing instruments, or be used to create entirely new instrument or controller configurations. Chemical sensor arrays and for other numerical image generation from computer processing or numeric simulation can be used to monitor or simulate natural physical phenomenon such as self-organizing process behavior or environmental conditions. Arrays of scalar or vector values are processed to extract pattern boundaries, geometric properties of pixels within the pattern boundaries (geometric center, weighted moments, etc.), and higher-level derived information (senses of rotation, segmented regions, pattern classification, syntax, grammars, sequences, etc.) which are used to create control signals for external audio, visual, and control equipment or algorithms. The invention also provides for MIDI and non-MIDI control signals.

Dependencies are inferred from the claim text.