# US 2021/0264636 A1 · Video analysis apparatus, person retrieval system, and person retrieval method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0264636 A1
- Document ID: US20210264636A1
- Status: Published application
- Kind: A1
- Application number: 17/253,842
- Filed: 2019-06-11
- Published: 2021-08-26
- Earliest claimed: 2018-07-18
- Assignees: Hitachi, Ltd.
- Applicants: Hitachi, Ltd.
- Inventors: Naoto AKIRA; Atsushi HIROIKE; Tsutomu IMADA; Hiromu NAKAMAE
- Web page: https://patentgrep.com/patent/US20210264636A1

## CPC

- G06T 7/292: Multi-camera tracking
- G06T 7/74: {involving reference images or patches}
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06V 40/168: {Feature extraction; Face representation}
- G06V 40/172: {Classification, e.g. identification}
- H04N 7/181: {for receiving images from a plurality of remote sources}
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06T 2207/30196: Human being; Person
- G06T 2207/30241: Trajectory
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A video analysis apparatus is connected to an image capturing apparatus including a plurality of cameras, in which the video analysis apparatus, for a plurality of persons in images captured by the cameras, tracks at least one of the plurality of persons, detects a plurality of parts of the tracked person, and on the basis of information defining scores used to determine a best shot of each of the parts, computes a score for the part for each of frames of videos from security cameras, compares, for each of the parts, the scores for each part computed for each of the frames to determine a best shot of each of the parts, stores, for each of the parts, a feature value in association with the part, and compares the feature values of each of the parts of the plurality of persons to determine identity of the persons.

## Claims

### Claim 1

A video analysis apparatus connected to an image capturing apparatus including a plurality of cameras, wherein
the video analysis apparatus,
for a plurality of persons in images captured by the cameras,
tracks at least one of the plurality of persons,
detects a plurality of parts of the tracked person,
for the plurality of parts detected, on a basis of information defining scores used to determine a best shot of each of the parts, computes a score for each part for each of frames of videos from the cameras,
compares, for each of the parts, the scores for each part computed for each of the frames to determine a best shot of each of the parts,
stores, for each of the parts, a feature value in association with each part, and
compares the feature values of each of the parts of the plurality of persons to determine identity of the persons.

### Claim 2 (depends on 1)

The video analysis apparatus according to claim 1, wherein
the video analysis apparatus detects at least two types of a head, a face, a set of clothes, a bag, a pair of shoes, a hat, and a watch of each of the persons as the plurality of parts.

### Claim 3 (depends on 1)

The video analysis apparatus according to claim 1, wherein
the video analysis apparatus selects, for each of the plurality of parts, items of an area, a frontal degree, a geometric feature, a luminance value, and a detection reliability, and uses an item score defined for each of the selected items to compute a score for each of the parts.

### Claim 4 (depends on 1)

The video analysis apparatus according to claim 1, wherein
the video analysis apparatus includes a storage section,
the storage section stores a parts DB in which each of the parts is associated with a feature value of each part, and
the parts DB stores, for one of the plurality of parts, a parts type representing a type of the part, the feature value of the part, and the score for the part in association with a parts ID of the part.

### Claim 5 (depends on 4)

The video analysis apparatus according to claim 4, wherein
the storage section, for one person trajectory of the person in an image captured by one of the plurality of cameras, stores a best shot camera table in which a person trajectory ID corresponding to the person trajectory is associated with the parts IDs of a plurality of parts in the person trajectory.

### Claim 6 (depends on 5)

The video analysis apparatus according to claim 5, wherein
the storage section stores a best shot merge table managing, for each part, the parts ID with a high score in a case where person trajectories of the plurality of persons in images captured by the plurality of cameras indicate an identical person.

### Claim 7

A person retrieval system comprising:
an image capturing apparatus including a plurality of cameras; and
a video analysis apparatus, wherein
the video analysis apparatus,
for a plurality of persons in images captured by the cameras,
tracks at least one of the plurality of persons,
detects a plurality of parts of the tracked person,
for the plurality of parts detected, on a basis of information defining scores used to determine a best shot of each of the parts, computes a score for the part for each of frames of videos from the cameras,
compares, for each of the parts, the scores for each part computed for each of the frames to determine a best shot of each of the parts,
stores, for each of the parts, a feature value in association with each part in a storage section, and
compares the feature values of each of the parts of the plurality of persons to determine identity of the persons.

### Claim 8 (depends on 7)

The person retrieval system according to claim 7, comprising:
a workstation connected to the video analysis apparatus, wherein
the workstation includes
a display section displaying a video captured by the image capturing apparatus, and
an input section receiving input information used to identify a person in the video displayed on the display section, and
the video analysis apparatus
computes feature values of a plurality of parts of the person identified by the input section of the workstation, and
retrieves the person identified by the input section on a basis of a best shot merge table stored in the storage section and allowing the feature value of each of the parts to be referenced.

### Claim 9 (depends on 7)

The person retrieval system according to claim 7, wherein
the video analysis apparatus identifies a region including an owner of baggage in a video from a baggage inspection apparatus, tracks a person trajectory of the identified person, acquires a best shot of each of the parts of the tracked person trajectory, compares the feature value of the acquired best shot of each of the parts with the feature value of each part stored in a corresponding parts DB of the best shot merge table stored in the storage section, and retrieves a person identical to the owner of the baggage.

### Claim 10

A person retrieval method of performing analysis using a video analysis apparatus connected to an image capturing apparatus including a plurality of cameras, wherein
the video analysis apparatus,
for a plurality of persons in images captured by the cameras,
tracks at least one of the plurality of persons,
detects a plurality of parts of the tracked person,
for the plurality of parts detected, on a basis of information defining scores used to determine a best shot of each of the parts, computes a score for the part for each of frames of videos from the cameras,
compares, for each of the parts, the scores for each part to determine a best shot of each of the parts,
stores, for each of the parts, a feature value in association with each part in a storage section, and
compares the feature values of each of the parts of the plurality of persons to determine identity of the persons.

### Claim 11 (depends on 10)

The person retrieval method according to claim 10, wherein
the video analysis apparatus selects, for each of the plurality of parts, items of an area, a frontal degree, a geometric feature, a luminance value, and a detection reliability, and uses an item score defined for each of the selected items to compute a score for each of the parts.

### Claim 12 (depends on 11)

The person retrieval method according to claim 11, wherein
the video analysis apparatus, for one of the plurality of parts, causes a parts type representing a type of the part, a feature value of the part, and the score for the part to be stored in the storage section in association with a parts ID of the part.

### Claim 13 (depends on 12)

The person retrieval method according to claim 12, wherein
the video analysis apparatus, in a case where person trajectories of a plurality of persons in images captured by the plurality of cameras indicate an identical person, selects, for each part, the parts ID with a high score, and manages a best shot merge table collectively including best shots of the parts of the identical person.

### Claim 14 (depends on 10)

The person retrieval method according to claim 10, wherein
the video analysis apparatus
displays a video captured by the image capturing apparatus, and when information used to identify the person in the displayed video is input to the video analysis apparatus, computes feature values of a plurality of parts of the identified person, and
retrieves the identified person on a basis of a best shot merge table stored in the storage section and allowing the feature value of each of the parts to be referenced.

### Claim 15 (depends on 10)

The person retrieval method according to claim 10, wherein
on a basis of an alert signal and a baggage video from a baggage inspection apparatus, the video analysis apparatus identifies a region including an owner of baggage, tracks a person trajectory of the owner, acquires a best shot of each of the parts of the tracked person trajectory, compares the feature value of the acquired best shot of each of the parts with the feature value of each of the parts in the best shot stored in a corresponding parts DB of the best shot merge table stored in the storage section, and retrieves a person identical to the owner of the baggage.
