# US 2003/0210807 A1 · Monitoring device, monitoring method and program for monitoring

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2003/0210807 A1
- Document ID: US20030210807A1
- Status: Published application
- Kind: A1
- Application number: 10/427,486
- Filed: 2003-05-01
- Published: 2003-11-13
- Earliest claimed: 2002-05-09
- Assignees: Matsushita Electric Industrial Co., Ltd.
- Inventors: Satoshi Sato; Masamichi Nakagawa; Kunio Nobori
- Web page: https://patentgrep.com/patent/US20030210807A1

## CPC

- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06T 7/215: Motion-based segmentation
- G06T 7/269: using gradient-based methods
- G08G 1/165: {for passive traffic, e.g. including static obstacles, trees}
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30261: Obstacle
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

In a monitoring device for vehicles, an optical flow detection section determines an optical flow from an image taken with a camera. A background flow estimation section determines a background flow, which is a flow for the camera image under the assumption that the image is a background, based on the motion of the vehicle. An approaching object detection section detects motion of an object in the surroundings of the vehicle by comparing the optical flow with the background flow.

## Claims

### Claim 1

A monitoring device comprising:
first means for determining an optical flow from an image taken with a camera capturing the surroundings of a vehicle;
second means for determining a background flow, which is an optical flow for the image under the assumption that the image is a background, based on motion of the vehicle; and
third means for detecting motion of an object existing in the surroundings of the vehicle by comparing the optical flow with the background flow.

### Claim 2 (depends on 1)

The monitoring device of claim 1, wherein the second means determines the background flow using a space model modeling a space the camera is capturing.

### Claim 3 (depends on 2)

The monitoring device of claim 2, wherein the space model is prepared based on distance data of objects the camera is capturing.

### Claim 4 (depends on 3)

The monitoring device of claim 3, wherein the vehicle is provided with an obstacle sensor, and the distance data is measured with the obstacle sensor.

### Claim 5 (depends on 2)

The monitoring device of claim 2, wherein the space model at least includes a road surface model modeling a road surface on which the vehicle is running.

### Claim 6 (depends on 2)

The monitoring device of claim 2, wherein the space model at least includes a wall model assuming a wall normal to a road surface on which the vehicle is running.

### Claim 7 (depends on 6)

The monitoring device of claim 6, wherein the wall is assumed to exist in the rear from the vehicle on a side.

### Claim 8 (depends on 1)

The monitoring device of claim 1, wherein the third means determines whether or not the magnitude of the optical flow is larger than a predetermined value, and
if the magnitude is larger than the predetermined value, the comparison is made with use of an angle difference, or otherwise the comparison is made without use of an angle difference.

### Claim 9 (depends on 8)

The monitoring device of claim 8, wherein the predetermined value is set according to the magnitude of the background flow at a corresponding position on the image.

### Claim 10 (depends on 1)

The monitoring device of claim 1, wherein the third means performs the steps of:
specifying approaching object candidate flows among optical flows by comparing the optical flows with background flows;
generating an approaching object candidate region by associating neighboring ones of the approaching object candidate flows with each other; and
determining the approaching object candidate flows related to the approaching object candidate region as noise when the area of the approaching object candidate region is smaller than a predetermined value.

### Claim 11

A monitoring device comprising:
first means for determining an optical flow from an image taken with a camera capturing the surroundings of a vehicle;
second means for determining a space flow, which is motion of a point on the image in a real world coordinate system, based on the optical flow, motion of the vehicle and a space model modeling a space the camera is capturing; and
third means for detecting motion of an object existing in the surroundings of the vehicle based on the space flow.

### Claim 12

A monitoring method comprising the steps of:
determining an optical flow from an image taken with a camera capturing the surroundings of a vehicle;
determining a background flow, which is an optical flow for the image under the assumption that the image is a background, based on motion of the vehicle; and
detecting motion of an object existing in the surroundings of the vehicle by comparing the optical flow with the background flow.

### Claim 13 (depends on 12)

The monitoring method of claim 12, further comprising the step of estimating motion of the vehicle using a vehicle speed sensor and a steering angle sensor provided for the vehicle.

### Claim 14

A program for monitoring, allowing a computer to execute the steps of:
determining an optical flow from an image taken with a camera capturing the surroundings of a vehicle;
determining a background flow, which is an optical flow for the image under the assumption that the image is a background, based on motion of the vehicle; and
detecting motion of an object existing in the surroundings of the vehicle by comparing the optical flow with the background flow.
