Generation device, generation method, and generation recording medium

A generation method performed by a generation device that includes a processor that executes instructions stored in a non-transitory computer readable medium and a storage device that comprises the non-transitory computer readable medium storing the instructions and is capable of accessing a set of training data sets each being a combination of a sound signal indicating a state and an explanatory sentence explaining the state in a character string, the processor performing:

• signal encoding processing of encoding, based on a first learning parameter, the sound signal to generate a sound feature vector;
• language encoding processing of encoding, based on a second learning parameter, the explanatory sentence to generate a language feature vector;
• language decoding processing of decoding, based on a third learning parameter, the sound feature vector into a text indicating the state;
• and
• update processing of updating the first learning parameter and the second learning parameter by contrast learning using a combination of a sound feature vector generated by the signal encoding processing and a language feature vector generated by the language encoding processing, and updates the third learning parameter based on a difference between the explanatory sentence and the text indicating the state decoded by the language decoding processing.

A non-transitory computer readable medium including instructions associated with a generation device that includes the processor that executes the instructions and a storage device that stores the instructions and is capable of accessing a set of training data sets each being a combination of a sound signal indicating a state and an explanatory sentence explaining the state in a character string, the non-transitory computer readable medium causing the processor to perform:

• signal encoding processing of encoding, based on a first learning parameter, the sound signal to generate a sound feature vector;
• language encoding processing of encoding, based on a second learning parameter, the explanatory sentence to generate a language feature vector;
• language decoding processing of decoding, based on a third learning parameter, the sound feature vector into a text indicating the state;
• and
• update processing of updating the first learning parameter and the second learning parameter by contrast learning using a combination of a sound feature vector generated by the signal encoding processing and a language feature vector generated by the language encoding processing, and updates the third learning parameter based on a difference between the explanatory sentence and the text indicating the state decoded by the language decoding processing.