# US 2016/0239714 A1 · Passenger counting device, passenger counting method, and program recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0239714 A1
- Document ID: US20160239714A1
- Status: Published application
- Kind: A1
- Application number: 15/025,968
- Filed: 2014-10-01
- Published: 2016-08-18
- Earliest claimed: 2013-10-09
- Assignees: NEC Corporation
- Applicants: NEC Corporation
- Inventors: Ryoma OAMI; Yukie EBIYAMA
- Web page: https://patentgrep.com/patent/US20160239714A1

## CPC

- G06V 20/593: {Recognising seat occupancy}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/73: using feature-based methods
- G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30196: Human being; Person
- G06T 2207/30201: Face
- G06T 2207/30232: Surveillance
- G06T 2207/30236: Traffic on road, railway or crossing
- G06T 2207/30242: Counting objects in image
- G06T 2207/30268: Vehicle interior
- G06V 20/54: of traffic, e.g. cars on the road, trains or boats
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Disclosed are a passenger counting device, etc., for accurately counting the number of occupants in a vehicle, including persons seated in the rear seats thereof. This vehicle passenger counting device comprises: a specific part detection means for detecting a specific part of a vehicle from acquired images and generating specific part detection results; an association means for associating the specific part detection results across images in order to compute a degree of movement of the vehicle and generate vehicle movement information including the result of the said computation; a person detection means for detecting persons who are occupants of the vehicle in the images and generating person detection results including location information of the detected persons; and an integration means for determining the number of occupants by integrating the person detection results on the basis of the vehicle movement information.

## Claims

### Claim 1

A passenger counting device comprising:
a memory storing instructions; and
at least one processor configured to process the instructions to:
detect a specific part of a vehicle in an image and generating specific-part detection results;
calculate a movement amount of the vehicle by associating the specific-part detection results with each other in images, and generating vehicle movement information including a result of the calculation;
detect a person who is a passenger, in the images and generating person detection results including position information of the detected person; and
integrate the person detection results based on the vehicle movement information and thereby determine number of passengers.

### Claim 2 (depends on 1)

The passenger counting device according to claim 1, wherein the processor configured to process the instructions to:
detect a wheel of the vehicle as the specific part of the vehicle.

### Claim 3 (depends on 2)

The passenger counting device according to claim 2, wherein the processor configured to process the instructions to:
detect the wheel in the acquired image and generating wheel position information indicating a position of the wheel in the image and
analyze the image based on the wheel position information, estimating whether the detected wheel is a front wheel or a rear wheel of the vehicle, and generating the specific-part detection results including a result of the estimation, and
associate wheels detected in the images with each other by using the results of the estimation included in the specific-part detection results.

### Claim 4 (depends on 3)

The passenger counting device according to claim 3, wherein the processor configured to process the instructions to:
obtain an edge feature or a gradient feature near the detected wheel based on the wheel position information, calculates likelihood that the detected wheel is a front wheel of the vehicle and likelihood that the detected wheel is a rear wheel of the vehicle, based on the edge feature or the gradient feature, and generates the specific-part detection results including a result of the calculation.

### Claim 5 (depends on 1)

The passenger counting device according to claim 1, wherein the processor configured to process the instructions to:
group the person detection results for persons located at positions close to each other, each of the positions being indicated by the corresponding position information for each of the images, and generating person grouping information indicating a result of the grouping,
generate group association information indicating association between each group in each of the images and a group in each different one of the images, based on the vehicle movement information and the person grouping information, and
determine number of persons seated in seats in each row of the vehicle based on the group association information and calculating number of passengers by integrating the numbers of persons in the respective rows.

### Claim 6 (depends on 1)

The passenger counting device according to claim 1, wherein the processor configured to process the instructions to:
estimate geometric distortion in the image, and
perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

### Claim 7 (depends on 1)

The passenger counting device according to claim 1, wherein the processor configured to process the instructions to:
estimate geometric distortion in the image, and
detect a person in the image by using a person detector corresponding to the estimated geometric distortion.

### Claim 8

A passenger counting method comprising:
detecting a specific part of a vehicle in an image and generating specific-part detection results;
calculating a movement amount of the vehicle by associating the specific-part detection results with each other in images, and generating vehicle movement information including a result of the calculation;
detecting a person who is a passenger, in the acquired images and generating person detection results including position information of the detected person; and
integrating the person detection results based on the vehicle movement information and thereby determining number of passengers.

### Claim 9

A non-transitory computer-readable program recording medium recording a passenger counting program causing a computer to execute:
a process of detecting a specific part of a vehicle in an image and generating specific-part detection results;
a process of calculating a movement amount of the vehicle by associating the specific-part detection results with each other in images, and generating vehicle movement information including a result of the calculation;
a process of detecting a person who is a passenger, in the acquired images and generating person detection results including position information of the detected person; and
a process of integrating the person detection results based on the vehicle movement information and thereby determining number of passengers.

### Claim 10 (depends on 2)

The passenger counting device according to claim 2, wherein the processor configured to process the instructions to:
group the person detection results for persons located at positions close to each other, each of the positions being indicated by the corresponding position information for each of the images, and generating person grouping information indicating a result of the grouping,
generate group association information indicating association between each group in each of the images and a group in each different one of the images, based on the vehicle movement information and the person grouping information, and
determine number of persons seated in seats in each row of the vehicle based on the group association information and calculating number of passengers by integrating the numbers of persons in the respective rows.

### Claim 11 (depends on 3)

The passenger counting device according to claim 3, wherein the processor configured to process the instructions to:
group the person detection results for persons located at positions close to each other, each of the positions being indicated by the corresponding position information for each of the images, and generating person grouping information indicating a result of the grouping,
generate group association information indicating association between each group in each of the images and a group in each different one of the images, based on the vehicle movement information and the person grouping information, and
determine number of persons seated in seats in each row of the vehicle based on the group association information and calculating number of passengers by integrating the numbers of persons in the respective rows.

### Claim 12 (depends on 4)

The passenger counting device according to claim 4, wherein the processor configured to process the instructions to:
group the person detection results for persons located at positions close to each other, each of the positions being indicated by the corresponding position information for each of the images, and generating person grouping information indicating a result of the grouping,
generate group association information indicating association between each group in each of the images and a group in each different one of the images, based on the vehicle movement information and the person grouping information, and
determine number of persons seated in seats in each row of the vehicle based on the group association information and calculating number of passengers by integrating the numbers of persons in the respective rows.

### Claim 13 (depends on 2)

The passenger counting device according to claim 2, wherein the processor configured to process the instructions to:
estimate geometric distortion in the image, and
perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

### Claim 14 (depends on 3)

The passenger counting device according to claim 3, wherein the processor configured to process the instructions to:
estimate geometric distortion in the image, and perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

### Claim 15 (depends on 4)

The passenger counting device according to claim 4, wherein the processor configured to process the instructions to:
estimate geometric distortion in the image, and
perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

### Claim 16 (depends on 5)

The passenger counting device according to claim 5, wherein the processor configured to process the instructions to:
estimate geometric distortion in the image, and
perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

### Claim 17 (depends on 2)

The passenger counting device according to claim 2, wherein the processor configured to process the instructions to:
estimate geometric distortion in the image, and
perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

### Claim 18 (depends on 3)

The passenger counting device according to claim 3, wherein the processor configured to process the instructions to:
estimate geometric distortion in the image, and
perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

### Claim 19 (depends on 2)

The passenger counting device according to claim 2, wherein the processor configured to process the instructions to:
estimate geometric distortion in the image, and
detect a person in the image by using a person detector corresponding to the estimated geometric distortion.

### Claim 20 (depends on 3)

The passenger counting device according to claim 3, wherein the processor configured to process the instructions to:
estimate geometric distortion in the image, and
detect a person in the image by using a person detector corresponding to the estimated geometric distortion.
