Multimodal human-robot interaction system for upper limb rehabilitation

A multimodal human-robot interaction system for upper limb rehabilitation, comprising an electroencephalography signal acquisition and processing module, a robot module, a comprehensive affected upper limb muscle signal acquisition and processing module, a rehabilitation training evaluation module, and a virtual reality module, wherein

• the electroencephalography signal acquisition and processing module is used for extracting a relevant feature parameter from an electroencephalography signal of a patient to obtain a motion intention of the patient, and then triggering a rehabilitation training action of the robot module;
• the robot module performs the training action under the triggering of the electroencephalography signal acquisition and processing module and a quantitative evaluation parameter fed back by the rehabilitation training evaluation module, so as to help an affected upper limb of the patient perform a spatial movement to complete a rehabilitation therapy;
• the comprehensive affected upper limb muscle signal acquisition and processing module is used for acquiring and obtaining a comprehensive assessment of the affected upper limb;
• the rehabilitation training evaluation module is used for processing and analyzing the comprehensive assessment of the affected upper limb, wherein the comprehensive assessment comprises a muscle function state index, a muscle strength index, and a muscle synergy index, respectively scoring the three indexes, and calculating a total score of the three indexes, so as to obtain a quantitative evaluation of the affected upper limb during rehabilitation training;
• and
• the virtual reality module is used for displaying a virtual environment for rehabilitation training, and increases motivation and engagement of the patient by providing feedback through multiple senses, wherein input parameters for the virtual reality module respectively come from the robot module and the rehabilitation training evaluation module,
• wherein the muscle strength index is reflected by force feedback data;
• the muscle function state index is reflected by feature parameters comprising an amplitude of a surface electromyography signal, an integrated electromyogram, a median frequency, and a mean frequency;
• and the muscle synergy index is reflected by posture and position information of the patient captured by machine vision,
• wherein respectively scoring the three indexes specifically comprise the following steps:
• obtaining the score of the muscle function state index by comparing feature values of surface electromyography signals of the patient and a healthy person, wherein the greater the difference between the two values is, the lower the score of the patient is;
• obtaining the score of the muscle strength index through force feedback data, wherein the greater the value of force is, the lower the active training degree of the patient is, and the lower the score of the patient is;
• obtaining the score of the muscle synergy index by comparing motion data of the patient and a healthy person and calculating errors of a motion velocity and a track of the affected upper limb, wherein the greater the errors are, the lower the score of the patient is;
• and
• obtaining the quantitative evaluation result of the upper limb rehabilitation training of the patient by calculating the total score of the three indexes, wherein the score is proportional to the motion ability and rehabilitation training effect of the affected upper limb.