Back to results
US 10,255,487 B2Granted
Emotion estimation apparatus using facial images of target individual, emotion estimation method, and non-transitory computer readable medium
AssigneeCasio Computer Co., Ltd.
InventorsTakashi Yamaya, Kouichi Nakagome, Katsuhiko Satoh
Life of this patent
Estimating the term…
Abstract
A speech determiner determines whether or not a target individual is speaking when facial images of the target individual are captured. An emotion estimator estimates the emotion of the target individual using the facial images of the target individual, on the basis of the determination results of the speech determiner.
Claims12
An emotion estimation apparatus comprising:
- a processor;
- and
- a memory that stores a first emotion recognition model and a second emotion recognition model, the first emotion recognition model and the second emotion recognition model estimating emotions;
- wherein the processor is configured to execute:
- a speech determination process of determining whether or not a target individual is speaking when facial images of the target individual are captured;
- an emotion recognition selection process of selecting the first emotion recognition model stored in the memory when the speech determination process has determined that the target individual is not speaking, and selecting the second emotion recognition model stored in the memory when the speech determination process has determined that the target individual is speaking;
- a facial characteristic value extraction process of extracting a first facial characteristic value including a mouth area of the target individual from a captured facial images of the target individual when the speech determination process has determined that the target individual is not speaking, and extracting a second facial characteristic value not including the mouth area of the target individual from the captured facial images of the target individual when the speech determination process has determined that the target individual is speaking;
- an emotion recognition process of estimating an emotion of the target individual using the first facial characteristic value extracted by the facial characteristic value extraction process and the first emotion recognition model selected by the emotion recognition selection process when the speech determination process has determined that the target individual is not speaking, and estimating an emotion of the target individual using the second facial characteristic value extracted by the facial characteristic value extraction process and the second emotion recognition model selected by the emotion recognition selection process when the speech determination process has determined that the target individual is speaking;
- and
- an emotion information output process of outputting emotion information showing the emotion of the target individual estimated by the emotion recognition process.
7 more elements
An emotion estimation method executed by an emotion estimation apparatus comprising a memory that stores a first emotion recognition model and a second emotion recognition model, the first emotion recognition model and the second emotion recognition model estimating emotions, the emotion estimation method including:
- a speech determination step of determining whether or not a target individual is speaking when facial images of the target individual are captured;
- an emotion recognition selection step of selecting the first emotion recognition model stored in the memory when the speech determination step has determined that the target individual is not speaking, and selecting the second emotion recognition model stored in the memory when the speech determination step has determined that the target individual is speaking;
- a facial characteristic value extraction step of extracting a first facial characteristic value including a mouth area of the target individual from a captured facial images of the target individual when the speech determination step has determined that the target individual is not speaking, and extracting a second facial characteristic value not including the mouth area of the target individual from the captured facial images of the target individual when the speech determination step has determined that the target individual is speaking;
- an emotion recognition step of estimating an emotion of the target individual using the first facial characteristic value extracted by the facial characteristic value extraction step and the first emotion recognition model selected by the emotion recognition selection step when the speech determination step has determined that the target individual is not speaking, and estimating an emotion of the target individual using the second facial characteristic value extracted by the facial characteristic value extraction step and the second emotion recognition model selected by the emotion recognition selection step when the speech determination step has determined that the target individual is speaking;
- and
- an emotion information output step of outputting emotion information showing the emotion of the target individual estimated by the emotion recognition step.
3 more elements
A non-transitory computer readable medium on which is recorded a program for causing a computer including an emotion estimation apparatus comprising a memory that stores a first emotion recognition model and a second emotion recognition model, the first emotion recognition model and the second emotion recognition model estimating emotions to execute:
- a speech determination process of determining whether or not a target individual is speaking when facial images of the target individual are captured;
- an emotion recognition selection process of selecting the first emotion recognition model stored in the memory when the speech determination process has determined that the target individual is not speaking, and selecting the second emotion recognition model stored in the memory when the speech determination process has determined that the target individual is speaking;
- a facial characteristic value extraction process of extracting a first facial characteristic value including a mouth area of the target individual from a captured facial images of the target individual when the speech determination process has determined that the target individual is not speaking, and extracting a second facial characteristic value not including the mouth area of the target individual from the captured facial images of the target individual when the speech determination process has determined that the target individual is speaking;
- an emotion recognition process of estimating an emotion of the target individual using the first facial characteristic value extracted by the facial characteristic value extraction process and the first emotion recognition model selected by the emotion recognition selection process when the speech determination process has determined that the target individual is not speaking, and estimating an emotion of the target individual using the second facial characteristic value extracted by the facial characteristic value extraction process and the second emotion recognition model selected by the emotion recognition selection process when the speech determination process has determined that the target individual is speaking;
- and
- an emotion information output process of outputting emotion information showing the emotion of the target individual estimated by the emotion recognition process.
3 more elements
Dependencies are inferred from the claim text.