Brain-computer interface based robotic arm self-assisting system and method

A robotic arm self-assisting method based on brain-computer interface, characterized in that the method comprises the steps as follows:

• 1) a user sits in front of a screen of a first computer, adjusts a position thereof, wears an electrode cap for an electroencephalogram acquisition, opens an electroencephalogram acquisition device and the first computer, and confirms that a signal acquisition status is good;
• 2) a brain-computer interface based robotic arm self-assisting system is started to confirm that a Microsoft Kinect vision sensor used for identifying and locating the user's mouth can correctly capture a user's mouth, and confirm that three preset cups to be taken are correctly placed in a field of view of the Microsoft Kinect vision sensor used for identifying and locating cups to be taken;
• 3) the screen of the first computer enters a function key interface of a flickering visual stimulation, the function key interface comprising four function keys of “cup1”, “cup2”, “cup3” and “back”;
• 4) the user gazes at one of the three function keys “cup1”, “cup2” or “cup3”, that is, selecting one of the three preset cups, and once a function key is selected, an intent of the user about a selection of a cup is obtained and sent to a visual identification and positioning module and a decision-making module;
• 5) the visual identification and positioning module identifies and locates a position of the cup and a position of the user's mouth based on the intent in the step 4) and sends, based on a TCP communication protocol, information of the positions of the cup selected by the user and the user's mouth to the decision-making module;
• 6) the decision-making module generates a corresponding execution instruction for the robotic arm based on the information of the positions of the cup and the user's mouth obtained in the step 5) and the intent obtained in the step 4), and sends the corresponding execution instruction for the robotic arm to a robotic arm control module;
• 7) the robotic arm control module performs a trajectory planning based on the execution instruction for the robotic arm and controls, based on a planned trajectory, the robotic arm to take the cup selected by the user and transfer the cup to the user's mouth;
• 8) after drinking water, the user gazes at the function key “back”, and once the function key is selected, an intent about returning the cup will be obtained and sent to the decision-making module;
• 9) the decision-making module generates, based on the intent for returning the cup obtained in the step 8), a corresponding execution instruction for the robotic arm, and sends the corresponding execution instruction for the robotic arm to the robotic arm control module;
• and
• 10) the robotic arm control module performs a trajectory planning based on the execution instruction for the robotic arm and controls, based on a planned trajectory, the robotic arm to return the cup selected by the user to an original position and restore an initial position status of the robotic arm, so as to realize a self-assisting function of the robotic arm for assisting the user to drink water.