Multi-modal modeling of temporal interaction sequences
AssigneeSRI International
InventorsAjay Divakaran, Behjat Siddiquie, Saad Khan, Jeffrey Lubin, Harpreet S. Sawhney
Life of this patent
Estimating the term…
Abstract
A multi-modal interaction modeling system can model a number of different aspects of a human interaction across one or more temporal interaction sequences. Some versions of the system can generate assessments of the nature or quality of the interaction or portions thereof, which can be used to, among other things, provide assistance to one or more of the participants in the interaction.
Claims37
A method for assessing an interaction involving at least two participants, at least one of the participants being a person, the method comprising, with a computing system:
- detecting, from multi-modal data captured by at least one sensing device during the interaction, a plurality of different behavioral cues expressed by the participants;
- analyzing the detected behavioral cues with respect to a plurality of different time scales, each of the time scales being defined by a time interval whose size can be compared to the size of other time intervals of the interaction;
- recognizing, based on the analysis of the detected behavioral cues, a temporal interaction sequence comprising a pattern of the behavioral cues corresponding to one or more of the time scales;
- and
- deriving, from the temporal interaction sequence, an assessment of the efficacy of the interaction.
A method for assessing an interaction involving at least two participants, at least one of the participants being a person, the method comprising, with a computing system:
- detecting, from multi-modal data captured by at least one sensing device, a plurality of different behavioral cues expressed by the participants during the interaction, the behavioral cues comprising one or more non-verbal cues and verbal content;
- recognizing a temporal interaction sequence comprising a pattern of the behavioral cues occurring over a time interval during the interaction;
- and
- deriving, from the temporal interaction sequence, an assessment of the efficacy of the interaction.
A method for assessing a person's emotional state during an interaction involving the person and at least one other participant, the method comprising, with a computing system:
- detecting, from multi-modal data captured by one or more sensing devices, a plurality of different behavioral cues expressed by the participants during the interaction;
- recognizing a plurality of temporal interaction sequences, each temporal interaction sequence comprising a pattern of the behavioral cues occurring over a time interval during the interaction, and at least one of the temporal interaction sequences involving behavioral cues of the person and behavioral cues of at least one other participant;
- assessing the person's emotional state during each of the temporal interaction sequences based on the behavioral cues involved in the temporal interaction sequence;
- determining a relative significance of each of the temporal interaction sequences to the interaction;
- and
- assessing the person's emotional state during the interaction as a whole based on the person's assessed emotional state during each of the temporal interaction sequences and the determined relative significance of each of the temporal interaction sequences to the interaction as a whole.
Dependencies are inferred from the claim text.