# US 2021/0082103 A1 · Analysis apparatus, analysis method, and storage mediumstorage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0082103 A1
- Document ID: US20210082103A1
- Status: Published application
- Kind: A1
- Application number: 17/095,460
- Filed: 2020-11-11
- Published: 2021-03-18
- Earliest claimed: 2018-02-27
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Jianquan LIU; Ka Wai YUNG
- Web page: https://patentgrep.com/patent/US20210082103A1

## CPC

- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 7/11: Region-based segmentation
- G06T 7/187: involving region growing; involving region merging; involving connected component labelling
- G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes}
- H04N 7/188: {Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253)}
- G06Q 50/265: {Personal security, identity or safety}
- G06T 2207/30242: Counting objects in image
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

The analysis apparatus (2000) includes a co-appearance event extraction unit (2020) and a frequent event detection unit (2040). The co-appearance event extraction unit (2020) extracts co-appearance events of two or more persons from each of a plurality of sub video frame sequences. The sub video frame sequence is included in a video frame sequence. The analysis apparatus (2000) may obtain the plurality of sub video frame sequences from one or more of the video frame sequences. The one or more of the video frame sequences may be generated by one or more of surveillance cameras. Each of the sub video frame sequences has a predetermined time length. The frequent event detection unit (2040) detects co-appearance events of the same persons occurring at a frequency higher than or equal to a pre-determined frequency threshold.

## Claims

### Claim 1

An analysis apparatus comprising:
a co-appearance event extraction unit extracting a co-appearance event of two or more persons from each of a plurality of sub video frame sequences, the sub video frame sequence being included in a video frame sequence generated by a surveillance camera, each of the sub video frame sequences having a predetermined time length; and
a frequent event detection unit detecting co-appearance events of same persons occurring at a frequency higher than or equal to a pre-determined frequency threshold,
wherein at least one the co-appearance event specifies that the unspecified combination of persons appeared in sequence in the video frame sequence among the predetermined time length and that the person appeared in the video frame sequence before the another person appeared in the video frame sequence.

### Claim 2 (depends on 1)

The analysis apparatus according to claim 1, further comprising an output unit outputting information indicating the co-appearance events of the same persons occurring at a frequency higher than or equal to the pre-determined frequency threshold.

### Claim 3 (depends on 1)

The analysis apparatus according to claim 1, wherein the co-appearance event extraction unit assigns a label to each of feature-values extracted from video frames of the video frame sequences, the feature-value indicating a feature of a person, feature-values indicating features of the same person being assigned the same label, feature-values indicating features of different persons being assigned different labels, and
wherein the frequent event detection unit handles the co-appearance events of a first label and a second label as the co-appearance events of a first person and a second person, the first label being assigned to a feature-value indicating a feature of the first person, the second label being assigned to a feature-value indicating a feature of the second person.

### Claim 4 (depends on 3)

The analysis apparatus according to claim 3, wherein the co-appearance event extraction unit assigns labels to the feature-values by repeatedly performing for each of the feature-values:
determining whether or not there is a feature-value to which a label is already assigned and which is similar to a current feature-value;
when there is no feature-value to which a label is already assigned and which is similar to a current feature-value, assigning a new label to the current feature-value; and
when there is a feature-value to which a label is already assigned and which is similar to a current feature-value, assigning to the current feature-value the same label with a feature-value most similar to the current feature-value.

### Claim 5 (depends on 3)

The analysis apparatus according to claim 3, wherein the frequent event detection unit performs:
for each sub video frames, calculating a size-N (N is an integer greater than 0) combinations of labels assigned to the feature-values extracted from the sub video frame:
counting the number of each size-N combinations included in the sub video frames; and
detecting the size-N combinations of the same labels the number of which is greater than or equal to the pre-determined frequency threshold, as the co-appearance events of the same persons occurring at a frequency higher than or equal to the pre-determined frequency threshold.

### Claim 6

An analysis method comprising:
extracting a co-appearance event of two or more persons from each of a plurality of sub video frame sequences, the sub video frame sequence being included in a video frame sequence generated by a surveillance camera, each of the sub video frame sequences having a predetermined time length; and
detecting co-appearance events of same persons occurring at a frequency higher than or equal to a pre-determined frequency threshold,
wherein at least one the co-appearance event specifies that the unspecified combination of persons appeared in sequence in the video frame sequence among the predetermined time length and that the person appeared in the video frame sequence before the another person appeared in the video frame sequence.

### Claim 7 (depends on 1)

The analysis method according to claim 1, further comprising outputting information indicating the co-appearance events of the same persons occurring at a frequency higher than or equal to the pre-determined frequency threshold.

### Claim 8 (depends on 1)

The analysis method according to claim 1, further comprising:
assigning a label to each of feature-values extracted from video frames of the video frame sequences, the feature-value indicating a feature of a person, feature-values indicating features of the same person being assigned the same label, feature-values indicating features of different persons being assigned different labels; and
handling the co-appearance events of a first label and a second label as the co-appearance events of a first person and a second person, the first label being assigned to a feature-value indicating a feature of the first person, the second label being assigned to a feature-value indicating a feature of the second person.

### Claim 9 (depends on 8)

The analysis method according to claim 8, wherein assigning labels to the feature-values by repeatedly performing for each of the feature-values comprises:
determining whether or not there is a feature-value to which a label is already assigned and which is similar to a current feature-value;
when there is no feature-value to which a label is already assigned and which is similar to a current feature-value, assigning a new label to the current feature-value; and
when there is a feature-value to which a label is already assigned and which is similar to a current feature-value, assigning to the current feature-value the same label with a feature-value most similar to the current feature-value.

### Claim 10 (depends on 8)

The analysis method according to claim 8, further comprising:
for each sub video frames, calculating a size-N (N is an integer greater than 0) combinations of labels assigned to the feature-values extracted from the sub video frame:
counting the number of each size-N combinations included in the sub video frames; and
detecting the size-N combinations of the same labels the number of which is greater than or equal to the pre-determined frequency threshold, as the co-appearance events of the same persons occurring at a frequency higher than or equal to the pre-determined frequency threshold.

### Claim 11

An non-transitory computer-readable storage medium storing a program that causes a processor to perform:
extracting a co-appearance event of two or more persons from each of a plurality of sub video frame sequences, the sub video frame sequence being included in a video frame sequence generated by a surveillance camera, each of the sub video frame sequences having a predetermined time length; and
detecting co-appearance events of same persons occurring at a frequency higher than or equal to a pre-determined frequency threshold,
wherein at least one the co-appearance event specifies that the unspecified combination of persons appeared in sequence in the video frame sequence among the predetermined time length and that the person appeared in the video frame sequence before the another person appeared in the video frame sequence.

### Claim 12 (depends on 11)

The storage medium according to claim 11, further comprising outputting information indicating the co-appearance events of the same persons occurring at a frequency higher than or equal to the pre-determined frequency threshold.

### Claim 13 (depends on 11)

The storage medium according to claim 11, further comprising:
assigning a label to each of feature-values extracted from video frames of the video frame sequences, the feature-value indicating a feature of a person, feature-values indicating features of the same person being assigned the same label, feature-values indicating features of different persons being assigned different labels; and
handling the co-appearance events of a first label and a second label as the co-appearance events of a first person and a second person, the first label being assigned to a feature-value indicating a feature of the first person, the second label being assigned to a feature-value indicating a feature of the second person.

### Claim 14 (depends on 13)

The storage medium according to claim 13, wherein assigning labels to the feature-values by repeatedly performing for each of the feature-values comprises:
determining whether or not there is a feature-value to which a label is already assigned and which is similar to a current feature-value;
when there is no feature-value to which a label is already assigned and which is similar to a current feature-value, assigning a new label to the current feature-value; and
when there is a feature-value to which a label is already assigned and which is similar to a current feature-value, assigning to the current feature-value the same label with a feature-value most similar to the current feature-value.

### Claim 15 (depends on 13)

The storage medium according to claim 13, further comprising:
for each sub video frames, calculating a size-N (N is an integer greater than 0) combinations of labels assigned to the feature-values extracted from the sub video frame:
counting the number of each size-N combinations included in the sub video frames; and
detecting the size-N combinations of the same labels the number of which is greater than or equal to the pre-determined frequency threshold, as the co-appearance events of the same persons occurring at a frequency higher than or equal to the pre-determined frequency threshold.
