US 8,219,438 B1Granted

Method and system for measuring shopper response to products based on behavior and facial expression

AssigneeVideoMining Corporation

InventorsHankyu Moon, Rajeev Sharma, Namsoon Jung

Life of this patent

Estimating the term…

Loading patent…

Abstract

The present invention is a method and system for measuring human response to retail elements, based on the shopper's facial expressions and behaviors. From a facial image sequence, the facial geometry—facial pose and facial feature positions—is estimated to facilitate the recognition of facial expressions, gaze, and demographic categories. The recognized facial expression is translated into an affective state of the shopper and the gaze is translated into the target and the level of interest of the shopper. The body image sequence is processed to identify the shopper's interaction with a given retail element—such as a product, a brand, or a category. The dynamic changes of the affective state and the interest toward the retail element measured from facial image sequence is analyzed in the context of the recognized shopper's interaction with the retail element and the demographic categories, to estimate both the shopper's changes in attitude toward the retail element and the end response—such as a purchase decision or a product rating.

Claims42

Dependent claims

A method for determining a person's response to a retail element, based on the person's facial expression and shopping behavior, comprising the following steps of:

a) detecting and tracking a face from first input images captured by at least a first means for capturing images, estimating two-dimensional and three-dimensional poses of the face, and localizing facial features, using at least a control and processing system,
b) estimating gaze direction of the person using the two-dimensional and three-dimensional poses and positions of the facial features and changes in affective state of the person by extracting emotion-sensitive features, and recognizing a demographic category of the person,
c) detecting and tracking the person from second input images captured by at least a second means for capturing images, producing a trajectory of the person, and estimating body orientation, using the control and processing system,
d) identifying the shopping behaviors of the person toward the retail element, utilizing position and the body orientation of the person relative to the retail element, and
e) determining intermediate responses and end response of the person to the retail element by analyzing the changes in affective states and interest, in the context of the shopping behavior and the demographics category of the person,
wherein the first means for capturing images and the second means for capturing images are connected to the control and processing system via at least a means for video interface, and wherein the shopping behaviors include showing interest, engagement, interaction, or purchasing.

3 more elements

An apparatus for determining a person's response to a retail element, based on the person's facial expression and shopping behavior, comprising:

a) at least a first means for capturing images that captures first input images,
b) at least a control and processing system that performs the following steps of:
detecting and tracking a face from the first input images, estimating two-dimensional and three-dimensional pose of the face, localizing facial features,
estimating gaze direction of the person using the two-dimensional and three-dimensional poses and the positions of the facial features and changes in affective state of the person by extracting emotion-sensitive features,
c) at least a second means for capturing images that captures second input images,
d) the control and processing system that performs the following steps of:
detecting and tracking the person from the second input images producing a trajectory of the person, and estimating body orientation,
identifying the shopping behaviors of the person toward the retail element, utilizing position and the body orientation of the person relative to the retail element, and
determining intermediate responses and end response of the person to the retail element by analyzing the changes in affective states and interest, in the context of the shopping behavior, and
e) at least a means for video interface that connects the first means for capturing images and the second means for capturing images to the control and processing system,
wherein the shopping behaviors include showing interest, engagement, interaction, or purchasing.

8 more elements

Dependencies are inferred from the claim text.

Facts

Application: 12/215,879Filed 30 Jun 2008
Granted: 10 Jul 2012
Term ends: Not estimatedThe source records don’t support an estimate.

Classification11 codes

G06Q 30/0201
Marketing; Price estimation or determination; Fundraising: market modelling; Market analysis; Collecting market data
G06F 18/295
Graphical models, e.g. Bayesian networks: markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding…
G06Q 30/0202
Market modelling; Market analysis; Collecting market data: market predictions or forecasting for commercial activities
G06Q 30/0204
Market modelling; Market analysis; Collecting market data: market segmentation