Information processing apparatus, information processing method, and storage medium

An information processing method comprising:

• extracting a plurality of features from data indicating a speech, wherein each of the plurality of features corresponds to a divided portion of the data indicating the speech, and wherein the extracting the plurality of features is performed using a neural network;
• encoding a series of chronological features of the data indicating the speech based on the plurality of features, wherein the encoding a series of chronological features of the data is performed using a neural network;
• generating information obtained by weighting features at a specific point in time associated with emotion classification, of the series of chronological features encoded, wherein the generating information is performed using a neural network;
• and
• classifying emotion and gender corresponding to the data indicating the speech using the information obtained by weighting the features at the specific point in time, wherein the classifying emotion and gender is performed using a neural network for the classification of emotion and a neural network for the classification of gender, the neural network for the classification of emotion and the neural network for the classification of gender being optimized through learning using an objective function which is calculated using a loss of the classification of emotion and a loss of the classification of gender.

A non-transitory computer-readable storage medium storing a program for causing a computer to function as each unit of an information processing apparatus, the information processing apparatus comprising:

• an extraction unit configured to extract a plurality of features from data indicating a speech, wherein each of the plurality of features corresponds to a divided portion of the data indicating the speech, and wherein the extraction unit is realized in a form of a neural network;
• an encoding unit configured to encode a series of chronological features of the data indicating the speech based on the plurality of features, wherein the encoding unit is realized in a form of a neural network;
• a generation unit configured to generate information obtained by weighting features at a specific point in time associated with emotion classification, of the series of chronological features encoded, wherein the generation unit is realized in a form of a neural network;
• and
• a classification unit configured to classify emotion and gender corresponding to the data indicating the speech using the information obtained by weighting the features at the specific point in time, wherein the classification unit comprises a neural network for the classification of emotion and a neural network for the classification of gender, the neural network for the classification of emotion and the neural network for the classification of gender being optimized through learning using an objective function which is calculated using a loss of the classification of emotion and a loss of the classification of gender.

A non-transitory computer-readable storage medium storing a recognition model performed by an information processing apparatus, the recognition model comprising:

• a first layer that performs convolution on an input value that is data indicating a speech, and makes output;
• a second layer that extracts time-series features of the data indicating the speech with an output value of the first layer as an input value, and makes output;
• a third layer that generates information obtained by weighting features at a specific point in time associated with emotion classification in an output value of the second layer by using self-attention, with the output value of the second layer as an input value, and makes output;
• and
• an output layer that outputs values indicating probabilities of emotion and gender based on an output value of the third layer, the output layer comprises a neural network for outputting the probability of emotion and a neural network for outputting the probability of gender, the neural network for outputting the probability of emotion and the neural network for outputting the probability of gender being optimized through learning using an objective function which is calculated using a loss of a classification of emotion and a loss of a classification of gender.