# US 2015/0178575 A1 · Three-dimensional object detection device and three-dimensional object detection method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0178575 A1
- Document ID: US20150178575A1
- Status: Published application
- Kind: A1
- Application number: 14/409,501
- Filed: 2013-07-25
- Published: 2015-06-25
- Earliest claimed: 2012-07-27
- Assignees: Nissan Motor Co., Ltd.
- Applicants: Nissan Motor Co., Ltd.
- Inventors: Osamu Fukata; Yasuhisa Hayakawa
- Web page: https://patentgrep.com/patent/US20150178575A1

## CPC

- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06T 7/285: using a sequence of stereo image pairs
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 20/653: {by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces}
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection}
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N 7/188: {Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position (signal generation from motion picture films H04N5/253)}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A three-dimensional object detection device has an image capturing unit, a three-dimensional object detection unit, a host vehicle speed detection unit, a light source detection unit and a controller. The image capturing unit captures images rearward of a vehicle. The three-dimensional object detection unit detects a presence of a three-dimensional object in a detection area, based on the captured images. The host vehicle speed detection unit detects a vehicle traveling speed. The light source detection unit detects a headlight light source of a headlight of another vehicle. The controller compares the traveling speeds of the object and the vehicle upon not detecting the headlight light source, and suppresses detection of the object upon determining one of the object traveling speed being equal to or less than the vehicle traveling speed, and a difference between the object and vehicle traveling speeds being less than a predetermined value.

## Claims

### Claim 1

A three-dimensional object detection device comprising:
an image capturing unit having a lens for capturing images of an area rearward of a host vehicle;
a three-dimensional object detection unit programmed to detect a presence of a three-dimensional object in a predetermined detection area, based on the images captured by the image capturing unit, and programmed to calculate a traveling speed of the three-dimensional object;
a host vehicle speed detection unit programmed to detect the traveling speed of the host vehicle;
a light source detection unit programmed to detect a headlight light source corresponding to a headlight of another vehicle, within a predetermined light source detection area that includes the detection area; and
a controller programmed to compare the traveling speed of the three-dimensional object and the traveling speed of the host vehicle upon the headlight light source not being detected by the light source detection unit, and programmed to perform a control process to suppress detection of the three-dimensional object upon determining one of the traveling speed of the three-dimensional object being equal to or less than the traveling speed of the host vehicle, and a difference between the traveling speed of the three-dimensional object and the traveling speed of the host vehicle being less than a predetermined value.

### Claim 2 (depends on 1)

The three-dimensional object detection device according to claim 1, further comprising
a darkness assessment unit programmed to carry out a darkness assessment process to access whether a darkness state exists,
the controller being programmed to perform the control process, in cases in which an assessment of an existence of the darkness state by the darkness assessment unit.

### Claim 3 (depends on 1)

The three-dimensional object detection device according to claim 1, further comprising
an image conversion unit programmed to perform viewpoint conversion of the images captured by the image capturing unit to bird's-eye view images, and
the three-dimensional object detection unit being further programmed to detect the presence of the three-dimensional object based on differential waveform information in which the differential waveform information is generated by aligning in bird's-eye view positions of the bird's-eye view images obtained at different points in time by the image capturing unit, and counting a number of pixels that exhibit a predetermined difference on a differential image of aligned bird's-eye view images to produce a frequency distribution, and the three-dimensional object detection unit being further programmed to calculate the traveling speed of the three-dimensional object based on temporal change of a waveform of the differential waveform information.

### Claim 4 (depends on 3)

The three-dimensional object detection device according to claim 3, wherein
the three-dimensional object detection unit being further programmed to count the number of pixels that exhibit the difference equal to or greater than a predetermined first threshold value on the differential image to produce the frequency distribution for generating the differential waveform information, and
the three-dimensional object detection unit being further programmed to detect the three-dimensional object based on the differential waveform information upon determining a peak value of the differential waveform information is equal to or greater than a predetermined second threshold value; and
the controller being further programmed to modify one of the the first threshold value and the second threshold value to a higher value, thereby suppressing detection of the three-dimensional object.

### Claim 5 (depends on 3)

The three-dimensional object detection device according to claim 3, wherein
during generation of the differential waveform information by the three-dimensional object detection unit, the controller is further programmed to suppress detection of the three-dimensional object by prompting the three-dimensional object detection unit to output a lower value from the process of counting the number of pixels that exhibit the predetermined difference on the differential image to produce the frequency distribution.

### Claim 6 (depends on 3)

The three-dimensional object detection device according claim 3, further comprising
a foreign matter detection unit programmed to extract a first local maximum from the differential waveform information generated at one or a plurality of first timings by the three-dimensional object detection unit, and acquire a reference value based on the first local maximum that was extracted; extract a second local maximum, corresponding to the first local maximum in the bird's-eye view images, from the differential waveform information newly generated at one or a plurality of second timings following the first timing, and acquire an evaluation target value based on the second local maximum in question; and detect whether foreign matter is deposited on the lens based on a change over time in the difference between the evaluation target value and the reference value;
the controller is further programmed to suppress detection of the three-dimensional object only upon the foreign matter detection unit having detected foreign matter deposited on the lens.

### Claim 7 (depends on 1)

The three-dimensional object detection device according to claim 1, further comprising
an image conversion unit programmed to perform viewpoint conversion of the images captured by the image capturing unit to bird's-eye view images, and
the three-dimensional object detection unit being further programmed to detect the presence of the three-dimensional object based on edge information detected from the bird's-eye view images obtained from the image conversion unit, and the three-dimensional object detection unit being further programmed to calculate the traveling speed of the three-dimensional object based on temporal change of the edge information.

### Claim 8 (depends on 7)

The three-dimensional object detection device according to claim 7, wherein
the three-dimensional object detection unit being further programmed to detect, from the bird's-eye view images, an edge component for which the luminance difference between adjacent pixel areas is equal to or greater than a first threshold value, and
the three-dimensional object detection unit being further programmed to detect the three-dimensional object based on the edge information upon determining an amount of the edge information based on the edge component is equal to or greater than a second threshold value,
the controller being further programmed to suppress detection of the three-dimensional object by modifying one of the first threshold value and the second threshold value to a higher value.

### Claim 9 (depends on 7)

The three-dimensional object detection device according to claim 7, wherein during detection of the edge information by the three-dimensional object detection unit, the controller is further programmed to suppress detection of the three-dimensional object by prompting the three-dimensional object detection unit to output a lower value of the edge information.

### Claim 10 (depends on 7)

The three-dimensional object detection device according to claim 7, further comprising
a foreign matter detection it programmed to extract a first local maximum from edge information including information about the edge lines generated at one or a plurality of first timings by the three-dimensional object detection unit, as well as acquiring a reference edge length based on the first local maximum in question; extract a second local maximum, corresponding to the first local maximum in the bird's-eye view images, from edge information newly generated at one or a plurality of second timings following the first timing, and acquire an evaluation edge length based on the second local maximum in question; and, based on a change over time in the difference between the evaluation edge length and the reference edge length, detects whether foreign matter is deposited on the lens;
the controller is further programmed to suppress detection of the three-dimensional object only upon the foreign matter detection unit having detected foreign matter deposited on the lens.

### Claim 11 (depends on 1)

The three-dimensional object detection device according to claim 1, wherein
the light source detection unit is further programmed to repeatedly detect light sources in the light source detection area, and, upon the headlight light source having been detected within a predetermined time interval, outputs a detection result indicating that the headlight light source was detected, and upon the headlight light source not being detected within the predetermined time interval, outputs a detection result indicating that the headlight light source was not detected.

### Claim 12 (depends on 2)

The three-dimensional object detection device according to claim 2, wherein
the darkness assessment unit is further programmed to repeatedly carry out the darkness assessment process for detecting light sources in an area different from the light source detection area, to thereby make the assessment as to the existence of the darkness state, and, upon a predetermined amount or more of light sources are detected within a predetermined assessment time interval, makes an assessment of the existence of the darkness state.

### Claim 13

A three-dimensional object detection method comprising:
performing viewpoint conversion to bird's-eye view images of images captured by an image capturing unit of an area rearward of a host vehicle;
generating differential waveform information from the bird's-eye view images taken at different points in time;
detecting a presence of a three-dimensional object in a predetermined detection area based on the differential waveform information;
determining whether the three-dimensional object is another vehicle by calculating a traveling speed of the three-dimensional object from temporal change of a waveform of the differential waveform information;
detecting headlight light source corresponding to a headlight of the other vehicle in a predetermined light source detection area that includes the predetermined detection area;
carrying out a darkness assessment process to access an existence of a darkness state, while making an assessment as to the existence of the darkness state, and in a case in which there is an assessment of the existence of the darkness state and additionally the headlight light source has not been detected, the traveling speed of the three-dimensional object and the traveling speed of the host vehicle are compared; and
suppressing an action deciding that the three-dimensional object is the other vehicle upon determining one of the traveling speed of the three-dimensional object is equal to or less than the traveling speed of the host vehicle, and a difference between the traveling speed of the three-dimensional object and the traveling speed of the host vehicle is less than a predetermined value.

### Claim 14

A three-dimensional object detection method comprising:
performing viewpoint conversion to bird's-eve view images of images captured by an image capturing unit of an area rearward of a host vehicle;
detecting edge information from the bird's-eye view images;
detecting a presence of a three-dimensional object in a predetermined detection area based on the edge information;
determining whether the three-dimensional object is another vehicle by calculating a traveling speed of the three-dimensional object from temporal change of a waveform of the edge information to decide;
detecting a headlight light source corresponding to a headlight of the other vehicle is in a predetermined light source detection area that includes the predetermined detection area;
carrying out a darkness assessment process to access an existence of a darkness state, while making an assessment as to the existence of the darkness state, and in a case in which there is an assessment of the existence of the darkness state and additionally the headlight light source has not been detected, the traveling speed of the three-dimensional object and the traveling speed of the host vehicle are compared; and
suppressing an action deciding that the three-dimensional object is the other vehicle upon determining one of the traveling speed of the three-dimensional object is equal to or less than the traveling speed of the host vehicle, and a difference between the traveling speed of the three-dimensional object and the traveling speed of the host vehicle is less than a predetermined value.
