Skeleton recognition method, computer-readable recording medium storing skeleton recognition program, skeleton recognition system, learning method, computer-readable recording medium storing learning program, and learning device

A non-transitory computer-readable storage medium storing a skeleton recognition program for causing a computer to perform processing, the processing comprising:

• acquiring, from a distance image of an object, a learning model that recognizes heat map images obtained by projecting likelihoods of a plurality of joint positions of the object from a plurality of directions;
• inputting a distance image to be processed to the learning model and acquiring heat map images in each of the plurality of directions;
• calculating three-dimensional coordinates regarding the plurality of joint positions of the object, using the heat map images in each of the plurality of directions and information that indicates a relative positional relationship of the plurality of directions;
• and
• outputting a skeleton recognition result that includes the three-dimensional coordinates regarding the plurality of joint positions.

A skeleton recognition system comprising:

• a learning device;
• and
• a recognition device, wherein
• the learning device includes
• a memory;
• and
• a processor coupled to the memory of the recognition device, the processor being configured to perform first processing, the first processing including:
• generating heat map images obtained by projecting likelihoods of a plurality of joint positions of an object from a plurality of directions, by using a distance image of the object captured in advance, and position information of a plurality of joints defined in advance;
• and
• learning a learning model that recognizes heat map images in each of the plurality of directions, by using training data that has the distance image of the object captured in advance as an explanatory variable, and the generated heat map images in the plurality of directions as objective variables, and
• the recognition device includes
• a memory;
• and
• a processor coupled to the memory of the recognition device, the processor of the recognition device being configured to perform second processing, the second processing including:
• acquiring the learning model;
• inputting a distance image to be processed to the learning model, and acquire the heat map images in each of the plurality of directions;
• calculating three-dimensional coordinates regarding positions of the plurality of joints of the object, using the heat map images in each of the plurality of directions and information that indicates a relative positional relationship of the plurality of directions;
• and
• outputting a skeleton recognition result that includes the three-dimensional coordinates regarding the positions of the plurality of joints.

A learning device comprising:

• a memory;
• and
• a processor coupled to the memory, the processor being configured to perform processing, the processing including:
• generating heat map images obtained by projecting likelihoods of a plurality of joint positions of an object from a plurality of directions, using a distance image of the object captured in advance, and position information of a plurality of joints defined in advance;
• and
• learning a learning model that recognizes heat map images in each of the plurality of directions, using training data that has the distance image of the object captured in advance as an explanatory variable, and the generated heat map images in the plurality of directions as objective variables.