Back to results
US 10,593,349 B2Granted
Emotional interaction apparatus
AssigneeThe George Washington University
InventorChung Hyuk Park
Life of this patent
Estimating the term…
Abstract
A system and method for emotional interaction. The system includes a robot that uses behavioral analysis automation to provide treatment and assessment of emotional communication and social skills for children with autism. The system generates a dataset including speech signals of one or more speakers, and assigns at least one of a set of labels to each of the speech signals for the one or more speakers. The set of labels includes at least three levels of emotional dimensions, the emotional dimensions include at least activation, valence, and dominance, and the at least three levels of emotional dimensions include a high state, a neutral state, and a low state.
Claims24
An emotional interaction system, comprising:
- an audio input device receiving speech signals from a speaker;
- a voice analyzer configured to:
- (A) during a configuration stage:
- receive configuration speech signals from the speaker by the audio input device, wherein a duration of the configuration speech signals is shorter than 5 minutes;
- extract configuration features from the configuration speech signals;
- and
- obtain configuration feature values for the speaker by calculating mean values of the configuration features;
- (B) during a test:
- receive the speech signals from the audio input device from the speaker;
- extract speech features from the received speech signals;
- obtain test speech features corresponding to the received speech signals by subtracting configuration feature values from the extracted speech features;
- determine at least one speech emotional dimensional label selected from a set of emotional dimension labels for each of the speaker signals for the speaker, wherein the set of emotional dimension labels includes at least one of activation, valence, and/or dominance;
- and
- using the test speech features, determine a speech emotional dimension level associated with at least one speech emotional dimension label corresponding to each of the speech signals, wherein the emotional dimension level includes a high state, a neutral state, and a low state;
- (C) during training:
- train a classifier for each of the set of emotional dimensional labels, using normalized training speech features of a plurality of speakers, wherein the normalized training speech features are obtained by a normalization process, the normalization process comprising:
- for each of the plurality of speakers,
- a) extracting speech features corresponding to all of the high state, the neutral state, and the low state associated with each emotional dimensional label,
- b) obtaining a subset of the extracted speech features, wherein the subset is randomly selected from the extracted speech features associated only with the neutral state of the at least one emotional dimensional label,
- c) calculating mean values of the subset of the extracted speech features, and
- d) obtaining the normalized training speech features by subtracting the calculated mean values from the extracted speech features associated with all of the high state, the neutral state, and the low state;
- an emotion controller configured to:
- receive the determined speech emotional dimension label and the determined speech emotional dimension level;
- and
- select a responsive emotional expression and/or responsive gestural behavior of the system based on the determined speech emotional dimension label and the determined speech emotional dimension level of the speaker, wherein the responsive emotional expression and/or responsive gestural behavior of the system being responsive to and different from the determined speech emotional dimension label and the determined speech emotional dimension level of the speaker;
- and
- an output device configured to present the selected responsive emotional expression and/or gestural behavior to the speaker.
24 more elements
A method for interacting with a speaker, comprising:
- receiving, by an audio input device, speech signals from the speaker;
- (A) during a configuration stage:
- receiving configuration speech signals from the speaker by the audio input device, wherein a duration of the configuration speech signals is shorter than 5 minutes;
- extracting configuration features from the configuration speech signals;
- and
- obtaining configuration feature values for the speaker by calculating mean values of the configuration features;
- (B) during a test:
- receiving the speech signals from the audio input device from the speaker;
- extracting the speech features from the received speech signals;
- obtaining test speech features corresponding to the received speech signals by subtracting a configuration feature set from the extracted features;
- determining at least one speech emotional dimensional label selected from a set of emotional dimension labels for each of the speaker signals for the speaker, wherein the set of emotional dimension labels includes at least one of activation, valence, and/or dominance;
- and
- using the test speech features, determining a speech emotional dimension level associated with at least one speech emotional dimension label corresponding to each of the speech signals, wherein the emotional dimension level includes a high state, a neutral state, and a low state:
- (C) during training:
- training a classifier for each of the set of emotional dimensional labels, using normalized training speech features of a plurality of speakers, wherein the normalized training speech features are obtained by a normalization process, the normalization process comprising:
- for each of the plurality of speakers,
- a) extracting speech features corresponding to all of the high state, the neutral state, and the low state associated with each emotional dimensional label,
- b) obtaining a subset of the extracted speech features, wherein the subset is randomly selected from the extracted features associated only with the neutral state of the at least one emotional dimensional label,
- c) calculating mean values of the subset of the extracted speech features, and
- d) obtaining the normalized speech features by subtracting the calculated mean values from the extracted speech features associated with all of the high state, the neutral state, and the low state;
- and
- presenting, by an output device, the selected responsive emotional expression and/or gestural behavior to the speaker.
19 more elements
Dependencies are inferred from the claim text.