# US 12,118,795 B2 · System and method for tracking customer movements in a customer service environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,795 B2
- Document ID: US12118795B2
- Status: Granted
- Kind: B2
- Application number: 17/260,802
- Filed: 2018-07-18
- Published: 2021-06-03
- Granted: 2024-10-15
- Earliest claimed: 2018-07-18
- Assignees: Tempo Analytics Inc.
- Applicants: Robert Laganière; Andrés Solís Montero; Luis E. Gurrieri
- Inventors: Robert Laganière; Andrés Solís Montero; Luis E. Gurrieri
- Web page: https://patentgrep.com/patent/US12118795B2

## CPC

- G06V 20/53: {Recognition of crowd images, e.g. recognition of crowd congestion}
- G06Q 30/0201: Market modelling; Market analysis; Collecting market data
- G06T 7/194: involving foreground-background segmentation
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06T 7/292: Multi-camera tracking
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06Q 20/4015: {using location information}
- G06Q 50/12: Hotels or restaurants
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30201: Face
- G06T 2207/30232: Surveillance
- G06T 2207/30241: Trajectory
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

System and methods for tracking transaction flow through a customer service area. The present invention provides automated, non-intrusive tracking of individuals, based on a series of still image frames obtained from one or more colour sensors. Images of individuals are extracted from each frame and the datasets resulting from the cropped images are compared across multiple frames. The datasets are grouped together into groups called “tracklets”, which can be further merged into “customer sets”. Various pieces of metadata related to individuals' movement (such as customer location and the duration of each transaction state) can be derived from the customer sets. Additionally, individual images may be anonymized into mathematical representations.

## Claims

### Claim 1

A system for detecting individuals within image frames and tracking individuals between said image frames, said system comprising:
processing circuitry, said processing circuitry being configured to receive an image frame; a server; and
a database, said database storing datasets, tracklets, and customer sets related to said individuals, each of said tracklets comprising at least one dataset, and each of said customer sets comprising at least one tracklet, and said database also storing metadata related to said individuals,
wherein said processing circuitry is further configured to process said image frame and to is olate at least one individual's image from said image frame,
and wherein said processing circuitry is further configured to generate a descriptor for said at least one individual's image,
and wherein said server receives a specific dataset including said descriptor and compares said specific dataset to a plurality of other datasets, said plurality of other datasets being comprised in a plurality of tracklets and each of said plurality of tracklets being designated as being in an active list, wherein said active list is managed as follows:
for each tracklet:
adding said tracklet to said active list at a first appearance of said tracklet;
assigning a time of last appearance to said tracklet, said time of last appearance corresponding to the last time an image of an individual represented by said tracklet was isolated;
determining an active list time for said tracklet based on a subtraction of said last time of appearance from a current time;
comparing said active list time to a limit time;
when said active list time is larger than said limit time, executing a merging step to merge said tracklet with another tracklet in said active list; and
when said merging step is unsuccessful, removing said tracklet from said active list,
and wherein said server calculates a similarity score between said dataset and each of said plurality of other datasets, producing a plurality of similarity scores.

### Claim 2 (depends on 1)

The system of claim 1, wherein one of said plurality of similarity scores meets a threshold and said dataset is added to a tracklet related to said one of said plurality of similarity scores.

### Claim 3 (depends on 1)

The system of claim 1, wherein none of said similarity scores meet a threshold and a new tracklet is created to contain said dataset.

### Claim 4 (depends on 1)

The system of claim 1, wherein said server further calculates a tracklet similarity between a pair of tracklets, and wherein said server merges said pair of tracklets to thereby produce a customer set when said tracklet similarity meets a threshold.

### Claim 5 (depends on 4)

The system of claim 4, wherein said server calculates tracking-related metadata based on said customer set.

### Claim 6 (depends on 1)

The system of claim 1, wherein said dataset comprises said descriptor, location information for said image, and time information for said image frame.

### Claim 7 (depends on 6)

The system of claim 6, wherein said similarity score is based on at least one of an appearance-similarity score based on said descriptor, and a proximity score based on said location information and said time information.

### Claim 8 (depends on 1)

The system of claim 1, wherein said descriptor includes a mathematical representation of said image.

### Claim 9 (depends on 1)

The system of claim 1, further comprising at least one camera, said at least one camera capturing said image frames and transmitting said image frames to said processing circuitry.

### Claim 10 (depends on 1)

The system of claim 1, wherein said server dynamically removes a background from said individual's image.