Multi-modal neural interfacing for prosthetic devices

A non-transitory computer-readable medium storing instructions, the instructions when executed by a processor cause the processor to:

• receive, using a plurality of types of sensors, a plurality of types of physiological activity signals from a prosthetic device user;
• determine, using a plurality of classifier modules, user movement states from each of the plurality of types of physiological activity signals, wherein each classifier module of the plurality of classifier modules is associated with a corresponding type of physiological activity signal;
• decode, using a plurality of decoders, movement intents from each of the plurality of types of physiological activity signals and from one or more of the user movement states, wherein each decoder of the plurality of decoders is associated with a corresponding type of physiological activity signal;
• and
• fuse, using a fusion module, the movement intents into a joint decision to control moveable elements of the prosthetic device, wherein the plurality of types of physiological activity signals include a combination of two or more of:
• a local field potential (LFP) signal,
• a unit activity (spike) signal,
• an epidural electrocorticography grid (ECoG) signal,
• an electromyography (EMG) signal,
• an electroencephalography (EEG) signal, and
• an electronystagmography (ENG) signal.