Method for dexterous hand movement decoding based on synergy basis manifold

A method for dexterous hand movement decoding based on synergy basis manifold, comprising:

• (1) acquiring electroencephalogram (EEG) signal feature and corresponding hand movement data, performing temporal alignment and preprocessing to obtain a training dataset;
• (2) constructing a dexterous hand movement decoding model based on a synergy basis manifold, comprising:
• a periodic network module, a control network module and a synergy basis recruitment module;
• wherein, the periodic network module is configured to map the EEG signal feature to a phase manifold, obtaining an EEG feature with periodic characteristics;
• the control network module is configured to extract decoupled spatiotemporal motion control parameters from the EEG feature generated by the periodic network module;
• based on the decoupled spatiotemporal motion control parameters obtained from the control network module, the synergy basis recruitment module recruits corresponding time-varying synergy basis from a maintained synergy basis dictionary, and the time-varying synergy basis are combined to decode to obtain the corresponding hand movements;
• wherein, the periodic network module is developed based on an autoencoder framework, comprising:
• a temporal convolutional layer and a deconvolution layer, a periodic encoding layer and a decoding layer based on Fourier transform, and a phase manifold mapping layer for periodic motion control feature extraction;
• a training of the periodic network module comprises a reconstruction loss for neural signal autoencoding;
• wherein, the control network module comprises two independent feedforward network layers, the two independent feedforward network layers map the EEG feature obtained from the periodic network module into the spatiotemporal motion control parameters containing an amplitude scaling factor and a time offset parameter;
• wherein, the control network module comprises a decoupling module for decoupling the spatiotemporal motion control parameters during a training phase, the decoupling module obtains motion parameters for independent control motion time and space by exchanging parameters between different samples in a batch;
• a training of the control network module comprises a decoupling loss;
• wherein, the synergy basis recruitment module maintains a synergy basis dictionary, the time-varying synergy basis are selected from the synergy basis dictionary based on the amplitude scaling factor and the time offset parameter obtained by the control network module;
• scaling an amplitude by using the corresponding amplitude scaling factor, selecting a time step of the synergy basis by using the time offset parameter, and obtaining the decoding result of hand movement by linearly combining all synergy basis;
• a training of the synergy basis recruitment module comprises a decoding loss of motion data;
• wherein, the synergy basis recruitment module maintains a synergy basis dictionary, during training phase, the synergy basis dictionary is used as a learnable parameter tensor, the parameters of the synergy basis dictionary and other parts of the dexterous hand movement decoding model are jointly estimated through backpropagation method;
• (3) constructing a composite loss function and training the dexterous hand movement decoding model by using the training dataset;
• (4) extracting EEG signal feature to be predicted, inputting the EEG signal feature to be predicted into the trained dexterous hand movement decoding model, and decoding to obtain corresponding hand movements.