Hand-eye, body part motion recognition and chronologically aligned display of recognized body parts

A motion recognition method comprising:

• acquiring time series information regarding motions of an operator;
• recognizing, on the basis of motion information that defines individual motions and the time series information, a motion of the operator corresponding to any one of the individual motions for each body part of the operator and generating a motion recognition result including a start time and an end time of the individual motion corresponding to the recognized motion of the operator for each body part of the operator, wherein the motion information comprises a corresponding relationship between the individual motions and motion elements in individual processes, the individual processes comprises an insertion process, a case fitting process, a soldering process and a screw fastening process;
• and
• displaying the motion recognition result of each body part in a chronologically aligned manner on the basis of the generated motion recognition result,
• wherein teacher data, the motion information and the time series information are had as inputs of a learned model, information indicating the motion of the operator corresponding to any one of the individual motions is had as an output of the learned model, the teacher data that defines a start time or an end time of an identified motion for each body part.

A non-transitory computer readable storage medium storing motion recognition program that causes a computer to:

• acquire time series information regarding motions of an operator;
• recognize, on the basis of motion information that defines individual motions and the time series information, a motion of the operator corresponding to any one of the individual motions for each body part of the operator, and generate a motion recognition result including a start time and an end time of the individual motion corresponding to the recognized motion of the operator for each body part of the operator, wherein the motion information comprises a corresponding relationship between the individual motions and motion elements in individual processes, the individual processes comprises an insertion process, a case fitting process, a soldering process and a screw fastening process;
• and
• display the motion recognition result of each body part in a chronologically aligned manner on the basis of the generated motion recognition result,
• the computer further comprising:
• a learned model configured to have, as inputs, teacher data, the motion information and the time series information and have, as an output, information indicating the motion of the operator corresponding to any one of the individual motions, the teacher data that defines a start time or an end time of an identified motion for each body part.

A motion recognition system comprising:

• a sensor;
• and
• a motion recognition apparatus,
• wherein the sensor includes a detection unit configured to detect motions of an operator and output time series information regarding the motions,
• the motion recognition apparatus includes
• a processor configured to:
• acquire the time series information,
• recognize, on the basis of motion information that defines individual motions and the time series information, a motion of the operator corresponding to any one of the individual motions for each body part of the operator, and generate a motion recognition result including a start time and an end time of the individual motion corresponding to the recognized motion of the operator for each body part of the operator, wherein the motion information comprises a corresponding relationship between the individual motions and motion elements in individual processes, the individual processes comprises an insertion process, a case fitting process, a soldering process and a screw fastening process, and
• a display configured to display the motion recognition result of each body part in a chronologically aligned manner on the basis of the generated motion recognition result,
• the motion recognition apparatus further comprising:
• a learned model configured to have, as inputs, teacher data, the motion information and the time series information and have, as an output, information indicating the motion of the operator corresponding to any one of the individual motions, the teacher data that defines a start time or an end time of an identified motion for each body part.