# US 2009/0268028 A1 · Flow line tracing system and program storage medium for supporting flow line tracing system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2009/0268028 A1
- Document ID: US20090268028A1
- Status: Published application
- Kind: A1
- Application number: 12/427,216
- Filed: 2009-04-21
- Published: 2009-10-29
- Earliest claimed: 2008-04-24
- Assignees: Toshiba Tec Kabushiki Kaisha
- Inventors: Tomonori Ikumi; Takashi Koiso; Naoki Sekine; Masami Takahata
- Web page: https://patentgrep.com/patent/US20090268028A1

## CPC

- H04N 7/181: {for receiving images from a plurality of remote sources}
- G06Q 30/06: Buying, selling or leasing transactions
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/173: {face re-identification, e.g. recognising unknown faces across different face tracks}
- G08B 13/1961: {Movement detection not involving frame subtraction, e.g. motion detection on the basis of luminance changes in the image}
- G08B 13/19613: {Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS

## Abstract

A flow line creation section generates flow line data indicative of a trajectory of a customer moving in a monitored area. The generated flow line data is stored in a flow line database. A customer extraction section extracts image data including the customer's face image from a video captured by a camera. The extracted image data is stored in a customer image database. A matching section matches the flow line data stored in the flow line database individually with the image data including the customer's face image corresponding to the flow line data, out of the image data stored in the customer image database. Data indicative of a correlation between the matched flow line data and image data is stored in a matching list database.

## Claims

### Claim 1

A flow line tracing system comprising:
flow line generating means for generating flow line data indicative of a trajectory of a customer moving in a monitored area;
flow line storage means for storing the flow line data generated by the flow line generating means;
image extraction means for extracting image data including the customer's face image from a video captured by a camera disposed so as to capture an image of the customer in a predetermined position within the monitored area;
image storage means for storing the image data extracted by the image extraction means;
matching means for matching the flow line data stored in the flow line storage means individually with the image data including the customer's face image corresponding to the flow line data, out of the image data stored in the image storage means; and
matching storage means for storing data indicative of a correlation between the flow line data and the image data matched by the matching means.

### Claim 2 (depends on 1)

A flow line tracing system according to claim 1, further comprising list display means for displaying a flow line list from which the flow line data stored in the flow line storage means is selectable, data selection means for selecting the image data matched with any of the flow line data selected from the flow line list, based on the data stored in the matching storage means, when the flow line data is selected, and analysis display means for displaying a customer image of the image data selected by the data selection means, together with a flow line of the flow line data selected from the flow line list.

### Claim 3 (depends on 1)

A flow line tracing system according to claim 1, further comprising list display means for displaying an image list from which the image data stored in the image storage means is selectable, data selection means for selecting the flow line data matched with any of the image data selected from the image list, based on the data stored in the matching storage means, when the image data is selected, and analysis display means for displaying a flow line of the flow line data selected by the data selection means, together with a customer image of the image data selected from the image list.

### Claim 4 (depends on 1)

A flow line tracing system according to claim 1, wherein the flow line storage means stores each flow line data together with data on the time when the customer corresponding to the flow line data is located in a predetermined position within the monitored area, the image storage means stores each image data together with data on the time when the image is captured, and the matching means matches flow line data and image data such that the difference between the respective time data thereof is the smallest.

### Claim 5 (depends on 4)

A flow line tracing system according to claim 4, further comprising time zone acceptance means for accepting input of a time zone, list display means for displaying a flow line list from which the flow line data stored together with the time data on the time zone of which the input is accepted by the time zone acceptance means is selectable, data selection means for selecting the image data matched with any of the flow line data selected from the flow line list, based on the data stored in the matching storage means, when the flow line data is selected, and analysis display means for displaying a customer image of the image data selected by the data selection means, together with a flow line of the flow line data selected from the flow line list.

### Claim 6 (depends on 4)

A flow line tracing system according to claim 4, further comprising time zone acceptance means for accepting input of a time zone, list display means for displaying an image list from which the image data stored together with the time data on the time zone of which the input is accepted by the time zone acceptance means is selectable, data selection means for selecting the image data matched with any of the image data selected from the image list, based on the data stored in the matching storage means, when the image data is selected, and analysis display means for displaying a flow line of the flow line data selected by the data selection means, together with a customer image of the image data selected from the image list.

### Claim 7

A flow line tracing system comprising:
flow line generating means for generating flow line data indicative of a trajectory of a customer moving in a monitored area from a video captured by first image capture means disposed so as to capture an image of the customer moving in the monitored area;
flow line storage means for storing the flow line data generated by the flow line generating means;
image extraction means for extracting image data including the customer's face image from a video captured by second image capture means disposed so as to capture an image of the customer in a predetermined position within the monitored area and configured to obtain an image clearer than that obtained by the first image capture means;
image storage means for storing the image data extracted by the image extraction means;
matching means for matching the flow line data stored in the flow line storage means individually with the image data including the customer's face image corresponding to the flow line data, out of the image data stored in the image storage means; and
matching storage means for storing data indicative of a correlation between the flow line data and the image data matched by the matching means.

### Claim 8 (depends on 7)

A flow line tracing system according to claim 7, wherein the first image capture means is a wide-angle camera.

### Claim 9 (depends on 7)

A flow line tracing system according to claim 7, wherein the second image capture means is a monitoring camera with a standard lens.

### Claim 10 (depends on 7)

A flow line tracing system according to claim 7, further comprising list display means for displaying a flow line list from which the flow line data stored in the flow line storage means is selectable, data selection means for selecting the image data matched with any of the flow line data selected from the flow line list, based on the data stored in the matching storage means, when the flow line data is selected, and analysis display means for displaying a customer image of the image data selected by the data selection means, together with a flow line of the flow line data selected from the flow line list.

### Claim 11 (depends on 7)

A flow line tracing system according to claim 7, further comprising list display means for displaying an image list from which the image data stored in the image storage means is selectable, data selection means for selecting the image data matched with any of the image data selected from the image list, based on the data stored in the matching storage means, when the image data is selected, and analysis display means for displaying a flow line of the flow line data selected by the data selection means, together with a customer image of the image data selected from the image list.

### Claim 12 (depends on 7)

A flow line tracing system according to claim 7, wherein the flow line storage means stores each flow line data together with data on the time when the customer corresponding to the flow line data is located in a predetermined position within the monitored area, the image storage means stores each image data together with data on the time when the image is captured, and the matching means matches flow line data and image data such that the difference between the respective time data thereof is the smallest.

### Claim 13 (depends on 12)

A flow line tracing system according to claim 12, further comprising time zone acceptance means for accepting input of a time zone, list display means for displaying a flow line list from which the flow line data stored together with the time data on the time zone of which the input is accepted by the time zone acceptance means is selectable, data selection means for selecting the image data matched with any of the flow line data selected from the flow line list, based on the data stored in the matching storage means, when the flow line data is selected, and analysis display means for displaying a customer image of the image data selected by the data selection means, together with a flow line of the flow line data selected from the flow line list.

### Claim 14 (depends on 12)

A flow line tracing system according to claim 12, further comprising time zone acceptance means for accepting input of a time zone, list display means for displaying an image list from which the image data stored together with the time data on the time zone of which the input is accepted by the time zone acceptance means is selectable, data selection means for selecting the image data matched with any of the image data selected from the image list, based on the data stored in the matching storage means, when the image data is selected, and analysis display means for displaying a flow line of the flow line data selected by the data selection means, together with a customer image of the image data selected from the image list.

### Claim 15 (depends on 7)

A flow line tracing system according to claim 7, wherein the image extraction means determines whether or not image data of the same person is already registered before the last human image data is obtained, determines whether or not the image quality of the last human image data is better than that of previous human image data when the image data of the same person is determined to be registered, and replaces the previous human image data with the last human image data when the image quality of the last human image data is determined to be better.

### Claim 16

A computer-readable storage medium stored with a program for supporting flow line tracing performed by a computer system, the program being configured to enable the computer system to fulfill:
a function to generate flow line data indicative of a trajectory of a customer moving in a monitored area;
a function to store a storage section of the computer system with the generated flow line data;
a function to extract image data including the customer's face image from a video captured by a camera disposed so as to capture an image of the customer in a predetermined position within the monitored area;
a function to store the storage section with the extracted image data;
a function to match the flow line data stored in the storage section individually with the image data including the customer's face image corresponding to the flow line data, out of the image data stored in the storage section; and
a function to store the storage section with data indicative of a correlation between the matched flow line data and image data.

### Claim 17 (depends on 16)

A storage medium according to claim 16, wherein the program enables the computer system to further fulfill a function to cause a display section of the computer system to display a flow line list from which the flow line data stored in the storage section is selectable, a function to select the image data matched with any of the flow line data selected from the flow line list, based on the data indicative of the correlation between the flow line data and the image data stored in the storage section, when the flow line data is selected, and a function to cause the display section to display a customer image of the selected image data, together with a flow line of the flow line data selected from the flow line list.

### Claim 18 (depends on 16)

A storage medium according to claim 16, wherein the program enables the computer system to further fulfill a function to cause a display section of the computer system to display an image list from which the image data stored in the storage section is selectable, a function to select the flow line data matched with any of the image data selected from the displayed image list, based on the data indicative of the correlation between the flow line data and the image data stored in the storage section, when the image data is selected, and a function to cause the display section to display a flow line of the selected flow line data, together with a customer image of the image data selected from the image list.
