Method and system for finding correspondence between point-of-sale data and customer behavior data

An apparatus for finding correspondence between retail point-of-sale data and shopper behavior data along with shopper track data from video observation, comprising:

• a) video cameras at point-of-purchase for collecting images of shoppers in each field of view,
• b) a computer and a shopper behavior analysis module that perform the following steps of:
• i. constructing a relational database of purchase items versus shoppers from the point-of-sale data,
• ii. collecting images of shoppers using cameras with fields of view at point-of-purchase,
• iii. triggering proximity and foreground object analysis of the images based on shopper orientation and time-based behavior detection,
• iv. running video analysis on the images to track the body and hands of shoppers,
• v. calculating the shopper behavior data based on information from tracking the shoppers, processed by the shopper behavior analysis module,
• vi. constructing a relational database of purchase items versus purchase events from the shopper behavior data,
• vii. incorporating event association constraints,
• viii. incorporating event time constraints using a linear equation, wherein the linear equation comprises a matrix that encodes relations between the purchase events and shoppers and a matrix that encodes time relations between the purchase events and checkout events, wherein the linear equation is
• Min
• ep
• [
• ∑
• ij
• ⁢
• {
• ip
• ij
• -
• ∑
• k
• ⁢
• ie
• ik
• ⁢
• ep
• kj
• }
• 2
• +
• α
• ⁢
• ∑
• j
• ⁢
• DistFit
• j
• +
• β
• ⁢
• ∑
• j
• ⁢
• LastStopFit
• j
• +
• λ
• ⁢
• ∑
• j
• ⁢
• VisualFit
• j
• wherein
• Min
• ep
• [
• ∑
• ij
• ⁢
• {
• ip
• ij
• -
• ∑
• k
• ⁢
• ie
• ik
• ⁢
• ep
• kj
• }
• 2
• represents an optimization problem for determining a likely event-person matrix,
• α
• ⁢
• ∑
• j
• ⁢
• DistFit
• j
• represents a constraint of distance between items,
• β
• ⁢
• ∑
• j
• ⁢
• LastStopFit
• j
• represents a constraint of distance between a closest item in a basket to a checkout register, and
• λ
• ⁢
• ∑
• j
• ⁢
• VisualFit
• j
• represents a constraint of the appearance of a shopper,
• ix. setting up a relational database of purchase events versus shoppers as unknown variables representing a correspondence,
• x. building a correspondence equation based on the relational database of purchase items versus shoppers, the relational database of purchase items versus purchase events, the relational database of purchase events versus shoppers, and the shopper track data,
• xi. solving the correspondence equation to find the correspondence, and
• xii. displaying correspondence data on a display screen to analyze the relationship the retail point-of-sale data and the shopper behavior data for the purpose of a media element rating system.