Method and apparatus for detecting keypoints of human body, electronic device and storage medium

An apparatus for detecting body keypoints, comprising;

• a processor;
• and
• a memory configured to store instructions executable by the processor,
• wherein the processor is configured to:
• extract, in response to detecting that an image comprises a body, two-dimensional (2D) coordinate data for identifying body keypoint positions in the image to obtain 2D pose data;
• and
• perform body keypoint feature fusion on the 2D pose data and depth data corresponding to the body keypoint positions to obtain three-dimensional (3D) pose data for identifying the body keypoint positions,
• wherein the processor is further configured to:
• acquire a first body motion state;
• describe, through first 3D pose data, a change of body keypoint positions corresponding to the first body motion state;
• and generate a first control instruction according to the first 3D pose data, and send the first control instruction to a receiving side device, so as to display an action simulating operation corresponding to the first body motion state on a display screen of the receiving side device,
• or
• acquire a second body motion state;
• describe, through second 3D pose data, a change of body keypoint positions corresponding to the second body motion state;
• compare the second 3D pose data with pre-configured pose data, and generate a second control instruction in response to an inconsistent comparison result;
• and send prompt information according to the second control instruction, so as to adjust the second body motion state to a target state according to the prompt information,
• or
• acquire a third body motion state;
• describe, through third 3D pose data, a change of body keypoint positions corresponding to the third body motion state;
• and send the third 3D pose data to the receiving side device, so as to display, on the display screen of the receiving side device, operations of a virtual avatar sampling the third 3D pose data.

A non-transitory computer-readable storage medium, having computer program instructions stored thereon, wherein the computer program instructions, when being executed by a processor, cause to implement operations, the operations comprising:

• extracting, in response to detecting that an image comprises a body, two-dimensional (2D) coordinate data for identifying body keypoint positions in the image to obtain 2D pose data;
• and
• performing body keypoint feature fusion on the 2D pose data and depth data corresponding to the body keypoint positions to obtain three-dimensional (3D) pose data for identifying the body keypoint positions,
• the operations further comprising:
• acquiring a first body motion state;
• describing, through first 3D pose data, a change of body keypoint positions corresponding to the first body motion state;
• and generating a first control instruction according to the first 3D pose data, and sending the first control instruction to a receiving side device, so as to display an action simulating operation corresponding to the first body motion state on a display screen of the receiving side device,
• or
• acquiring a second body motion state;
• describing, through second 3D pose data, a change of body keypoint positions corresponding to the second body motion state;
• comparing the second 3D pose data with pre-configured pose data, and generating a second control instruction in response to an inconsistent comparison result;
• and sending prompt information according to the second control instruction, so as to adjust the second body motion state to a target state according to the prompt information,
• or
• acquiring a third body motion state;
• describing, through third 3D pose data, a change of body keypoint positions corresponding to the third body motion state;
• and sending the third 3D pose data to the receiving side device, so as to display, on the display screen of the receiving side device, operations of a virtual avatar sampling the third 3D pose data.