Human-machine interaction method, system, processing device and computer readable storage medium

A processing device, comprising:

• a processor and a memory storing a computer program that is capable of running on the processor, wherein a human-machine interaction method is implemented when the processor executes the computer program, and the human-machine interaction method comprises:
• controlling a 3D display to output a 3D view, wherein a virtual target object is presented by the 3D view;
• receiving a user image taken by a sight tracking camera, and detecting an eye gaze region according to the user image;
• receiving a hand image taken by a gesture detection camera, and detecting whether a user's hand collides with the virtual target object and/or grabs the virtual target object according to the hand image;
• controlling playing of the 3D view according to whether human eyes gaze at the virtual target object, whether the user's hand collides with the virtual target object and grasps the virtual target object;
• wherein detecting whether the user's hand collides with the virtual target object and/or grabs the virtual target object according to the hand image comprises:
• preselecting a plurality of parts of the hand as key nodes of the hand;
• acquiring the hand image taken by the gesture detection camera, determining position data of the key nodes of the hand, and detecting whether the user's hand collides with the virtual target object according to a position of the virtual target object and the position data of the key nodes of the hand;
• and
• detecting whether the user's hand grabs the virtual target object when the user's hand collides with the virtual target object.

A non-transitory computer readable storage medium storing computer-executable instructions thereon, wherein the computer-executable instructions are used for executing a human-machine interaction method, which comprises:

• controlling a 3D display to output a 3D view, wherein a virtual target object is presented by the 3D view;
• receiving a user image taken by a sight tracking camera, and detecting an eye gaze region according to the user image;
• receiving a hand image taken by a gesture detection camera, and detecting whether a user's hand collides with the virtual target object and/or grabs the virtual target object according to the hand image;
• controlling playing of the 3D view according to whether human eyes gaze at the virtual target object, whether the user's hand collides with the virtual target object and grasps the virtual target object;
• wherein detecting whether the user's hand collides with the virtual target object and/or grabs the virtual target object according to the hand image comprises:
• preselecting a plurality of parts of the hand as key nodes of the hand;
• acquiring the hand image taken by the gesture detection camera, determining position data of the key nodes of the hand, and detecting whether the user's hand collides with the virtual target object according to a position of the virtual target object and the position data of the key nodes of the hand;
• and
• detecting whether the user's hand grabs the virtual target object when the user's hand collides with the virtual target object.