Machine learning device, estimation system, training method, and recording medium

A method for estimating emotion comprising:

• receiving, via a communication interface, a first code from a first measuring device worn on a foot portion of a user, the first code being generated by encoding first sensor data including time-series acceleration data and angular velocity data related to movement of the foot portion using a first encoding model;
• receiving, via the communication interface, a second code from a second measuring device worn on a wrist of the user, the second code being generated by encoding second sensor data including time-series pulse wave data measured from the wrist using a second encoding model;
• obtaining, from the second code, a time-series pulse signal representing the time-series pulse wave data;
• detecting pulse peaks from the time-series pulse signal and calculating a plurality of pulse intervals between the pulse peaks;
• performing frequency analysis on a time-series signal of the plurality of pulse intervals to extract a low-frequency component and a high-frequency component;
• calculating a ratio between the high-frequency component and the low-frequency component as an index indicating an activity state of an autonomic nervous function of the user;
• determining a wakefulness level of the user on the basis of at least a pulse rate calculated from the plurality of pulse intervals and the index indicating the activity state of the autonomic nervous function;
• determining a valence indicating pleasantness or unpleasantness of an emotional state of the user on the basis of a variation in the plurality of pulse intervals;
• inputting the first code representing a physical gait of the user and the second code representing a physiological state of the user, together with the determined wakefulness level and valence, to an estimation model having parameters learned in advance by machine learning using training data including sensor data and correct answer data;
• obtaining, from the estimation model, data representing one of four emotion categories consisting of “delight”, “anger”, “sadness”, and “pleasure” as an estimated emotional state of the user;
• and
• outputting control data indicating the estimated emotional state to a computing device so as to cause a change in an operational state of the computing device in accordance with the estimated emotional state.

A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:

• receiving, via a communication interface, a first code from a first measuring device worn on a foot portion of a user, the first code being generated by encoding first sensor data including time-series acceleration data and angular velocity data related to movement of the foot portion using a first encoding model;
• receiving, via the communication interface, a second code from a second measuring device worn on a wrist of the user, the second code being generated by encoding second sensor data including time-series pulse wave data measured from the wrist using a second encoding model;
• obtaining, from the second code, a time-series pulse signal representing the time-series pulse wave data;
• detecting pulse peaks from the time-series pulse signal and calculating a plurality of pulse intervals between the pulse peaks;
• performing frequency analysis on a time-series signal of the plurality of pulse intervals to extract a low-frequency component and a high-frequency component;
• calculating a ratio between the high-frequency component and the low-frequency component as an index indicating an activity state of an autonomic nervous function of the user;
• determining a wakefulness level of the user on the basis of at least a pulse rate calculated from the plurality of pulse intervals and the index indicating the activity state of the autonomic nervous function;
• determining a valence indicating pleasantness or unpleasantness of an emotional state of the user on the basis of a variation in the plurality of pulse intervals;
• inputting the first code representing a physical gait of the user and the second code representing a physiological state of the user, together with the determined wakefulness level and valence, to an estimation model having parameters learned in advance by machine learning using training data including sensor data and correct answer data;
• obtaining, from the estimation model, data representing one of four emotion categories consisting of “delight”, “anger”, “sadness”, and “pleasure” as an estimated emotional state of the user;
• and
• outputting control data indicating the estimated emotional state to a computing device so as to cause a change in an operational state of the computing device in accordance with the estimated emotional state.