Method, system and computer-readable storage medium for cross-task unseen emotion class recognition

A speech recognition device for unseen emotion class recognition, comprising:

• a memory having stored thereon a plurality of instructions;
• and
• a processor coupled to the memory, wherein the processor is configured to, when executing the instructions:
• receive, by an emotion recognition model, a speech sample to be tested;
• calculate, by an encoder, a sample embedding to be tested of the speech sample to be tested;
• calculate a first distance metric between the sample embedding to be tested and a first registered emotion category representation, and a second distance metric between the sample embedding to be tested and a second registered emotion category representation, wherein the second registered emotion category is not included in a plurality of basic emotion categories;
• comparing the first distance metric and the second distance metric;
• and
• determine an emotion category of the speech sample to be tested based on the comparison result,
• wherein the processor is further configured to:
• receive, by the emotion recognition model, a first training speech sample, wherein the first training speech sample indicates a first emotion category in the basic emotion categories, and the basic emotion categories include respectively a plurality of training speech samples;
• calculate, by the encoder, a first embedding of the first training speech sample;
• calculate respectively, by the encoder, a plurality of sample embeddings of the training speech samples of each of the basic emotion categories;
• calculate a center-of-mass representation of each of the basic emotion categories, wherein the center-of-mass representations are respectively averages of the sample embeddings of different basic emotion categories;
• calculate respectively a cosine similarity of the first embedding with the center-of-mass representation of each of the basic emotion categories, wherein an angular margin is added to the cosine similarity when calculating the cosine similarity with the first emotion category, and the angular margin is negative;
• calculate a loss according to a loss function;
• and
• adjust a plurality of parameters of the emotion recognition model based on the calculated loss, wherein the loss function is associated with the cosine similarity.

A computer-readable non-transitory storage medium for unseen emotion class recognition loaded with a computer-readable program capable of, after being read by a speech recognition device, configure the speech recognition device to:

• receive, by an emotion recognition model, a speech sample to be tested;
• calculate, by an encoder, a sample embedding to be tested of the speech sample to be tested;
• calculate a first distance metric between the sample embedding to be tested and a first registered emotion category representation, and a second distance metric between the sample embedding to be tested and a second registered emotion category representation, wherein the second registered emotion category is not included in a plurality of basic emotion categories;
• comparing the first distance metric and the second distance metric;
• and
• determine an emotion category of the speech sample to be tested based on the comparison result;
• wherein the computer-readable program is capable of, after being read by the speech recognition device, further configure the speech recognition device to:
• receive, by the emotion recognition model, a first training speech sample, wherein the first training speech sample indicates a first emotion category in the basic emotion categories, and the basic emotion categories include respectively a plurality of training speech samples;
• calculate, by the encoder, a first embedding of the first training speech sample;
• calculate respectively, by the encoder, a plurality of sample embeddings of the training speech samples of each of the basic emotion categories;
• calculate a center-of-mass representation of each of the basic emotion categories, wherein the center-of-mass representations are respectively averages of the sample embeddings of different basic emotion categories;
• calculate respectively a cosine similarity of the first embedding with the center-of-mass representation of each of the basic emotion categories, wherein an angular margin is added to the cosine similarity when calculating the cosine similarity with the first emotion category, and the angular margin is negative;
• calculating a loss according to a loss function;
• and
• adjusting a plurality of parameters of the emotion recognition model based on the calculated loss, wherein the loss function is associated with the cosine similarity.