# US 9,435,887 B2 · Object detecting apparatus, and method of operating the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,435,887 B2
- Document ID: US9435887B2
- Status: Granted
- Kind: B2
- Application number: 14/697,097
- Filed: 2015-04-27
- Published: 2016-04-28
- Granted: 2016-09-06
- Earliest claimed: 2014-10-23
- Assignees: Hyundai Mobis Co., Ltd.
- Applicants: Hyundai Mobis Co., Ltd.
- Inventors: Young shin Kim; Won gyum Kim; Soo yong Ryu
- Web page: https://patentgrep.com/patent/US9435887B2

## CPC

- G01S 17/06: Systems determining position data of a target
- B60T 7/22: initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle {, or by means of contactless obstacle detectors mounted on the vehicle}
- G01P 3/36: Devices characterised by the use of optical means, e.g. using infrared, visible, or ultraviolet light (G01P3/68 takes precedence; gyrometers using the Sagnac effect, i.e. rotation-induced shifts between counter-rotating electromagnetic beams G01C19/64)
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/931: of land vehicles
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- B60T 2201/022: Collision avoidance systems
- B60T 2210/32: Vehicle surroundings
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/30261: Obstacle
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- B60T: VEHICLE BRAKE CONTROL SYSTEMS OR PARTS THEREOF; BRAKE CONTROL SYSTEMS OR PARTS THEREOF, IN GENERAL (electrodynamic brake systems for vehicle, in general B60L; brakes per se, i.e. devices where braking effect occurs, including ultimate brake actuators, F16D); ARRANGEMENT OF BRAKING ELEMENTS ON VEHICLES IN GENERAL; PORTABLE DEVICES FOR PREVENTING UNWANTED MOVEMENT OF VEHICLES; VEHICLE MODIFICATIONS TO FACILITATE COOLING OF BRAKES
- G01P: MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT (measuring or recording blood flow A61B5/02, A61B8/06; monitoring speed or deceleration of electrically-propelled vehicles B60L3/00; vehicle lighting systems adapted to indicate speed B60Q1/54; determining position or course in navigation, measuring ground distance in geodesy or surveying G01C; combined measuring devices for measuring two or more variables of movement G01C23/00; measuring velocity of sound G01H; measuring velocity of light G01J7/00; measuring direction or velocity of solid objects by reception or emission of radiowaves or other waves and based on propagation effects, e.g. Doppler effect, propagation time, direction of propagation, G01S; measuring speed of nuclear radiation G01T; measuring acceleration of gravity G01V; {measuring or recording the speed of trains B61L23/00; speed indicators incorporated in motor vehicles B60K35/00; measuring frequency or phase G01R; traffic control G08G})
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An object detecting apparatus and an operating method thereof are disclosed. An apparatus for detecting an object includes: a sensor unit including a light distance and ranging (LiDAR) sensor for collecting LiDAR data and a camera sensor for collecting image data; an area dividing unit configured to divide a common detection area of the LiDAR sensor and the camera sensor into a plurality of areas; a data analyzing unit configured to analyze the LiDAR data to extract a first object information and/or analyze the image data to extract a second object information; and a sensor signal converging unit configured to determine whether a dangerous object exists for each of the divided areas based on the first object information and/or the second object information.

## Claims

### Claim 1

An apparatus for detecting an object, comprising:
a sensor unit including a light distance and ranging (LiDAR) sensor configured for collecting LiDAR data and a camera sensor configured for collecting image data;
an area dividing unit configured to divide a common detection area of the LiDAR sensor and the camera sensor into a plurality of areas;
a data analyzing unit configured to analyze the LiDAR data to extract a first object information and analyze the image data to extract a second object information;
a sensor signal converging unit configured to determine whether a dangerous object exists for each of the divided areas based on the first object information and the second object information; and
a vehicle information collecting unit configured to collect speed information about a vehicle,
wherein:
the plurality of areas divided by the area dividing unit comprise a ground non-included area and a ground included area;
the data analyzing unit is configured to insert relative speed information of an object against the vehicle into at least one of the first object information and the second information; and
the sensor signal converging unit is configured to determine that a dangerous object exists in the ground included-area when the first object information about the ground included is about a stopped object and the second object information about the stopped object is included in the image data.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the sensor signal converging unit comprises:
a coordinate converging module configured to convert the LiDAR data and the image data into data in a coordinate plane; and
an object comparing module configured to compare the first object information included in the LiDAR data and the second object information included in the image data in the coordinate plane,
wherein the sensing signal converging unit is configured to determine whether a dangerous objects exists for each of the divided areas based on a result of the comparison.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein when the first object information and the second object information have a difference less than a predetermined threshold value as a result of the comparison, the sensor signal converging unit is configured to determine that a first object and a second object are the same object.

### Claim 4 (depends on 2)

The apparatus of claim 2, wherein:
the first object information is center coordinates of a first object for the coordinate plane; and
the second object information is center coordinates of a second object for the coordinate plane.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein when the data analyzing unit extracts the first object information from the LiDAR data for the ground non-included area, the sensor signal converging unit is configured to determine that a dangerous object exists in the ground non-included area.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the sensor unit is configured to adjust an object detection threshold value of the camera sensor based on a predetermined sensitivity adjustment condition.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein:
the data analyzing unit is configured to insert relative speed information of an object against the vehicle into at least one of the first object information and the second object information; and
the sensor signal converging unit is configured to determine whether the first object information about the ground included area is related to a stopped object based on the relative speed information, and when it is determined that the first object information is related to the moving object, the sensor signal converging unit is configured to determine that a dangerous object exists in the ground included area.

### Claim 8 (depends on 1)

The apparatus of claim 1, further comprising a driving unit configured to operate an autonomous emergency braking (AEB) system of the vehicle when the sensor signal converging unit processes that the dangerous object exists.

### Claim 9 (depends on 1)

The apparatus of claim 1, further comprising a synchronizing unit configured to synchronize the LiDAR data transmitted from the LiDAR sensor and the image data transmitted from the camera sensor based on a time,
wherein the sensor signal converging unit is configured to determine whether a dangerous object exists for each of the divided areas based on the first object information and the second object information when the synchronization is completed.

### Claim 10

An apparatus for detecting an object, comprising:
a sensor unit comprising a light distance and ranging (LiDAR) sensor configured for collecting LiDAR data and a camera sensor configured for collecting image data;
an area dividing unit configured to divide a common detection area of the LiDAR sensor and the camera sensor into a plurality of areas;
a data analyzing unit configured to analyze the LiDAR data to extract a first object information and analyze the image data to extract a second object information;
a sensor signal converging unit configured to determine whether a dangerous object exists for each of the divided areas based on the first object information and the second object information; and
a vehicle information collecting unit configured to collect speed information about a vehicle,
wherein:
the plurality of areas divided by the area dividing unit comprise a ground non-included area and a ground included area;
the data analyzing unit is configured to insert relative speed information of an object against the vehicle into at least one of the first object information and the second object information;
the sensor signal converging unit is configured to determine whether the first object information about the ground included area is related to a stopped object based on the relative speed information, and when it is determined that the first object information is related to the moving object, the sensor signal converging unit is configured to determine that a dangerous object exists in the ground included area;
the data analyzing unit is configured to extract lane path and boundary information included in the image data; and
when the first object information for the ground included area is related to a stopped object existing out of the lane boundary and in the lane path, the sensor signal converging unit determines that a dangerous object exists in the ground included area.

### Claim 11

A method of operating an apparatus for detecting an object, comprising:
generating light distance and ranging LiDAR data by using a LiDAR sensor;
generating image data by using a camera sensor;
dividing a common detection area of the LiDAR sensor and the camera sensor into a ground non-included area and a ground included area;
analyzing at least one of the LiDAR data to extract first object information and the image data to extract second object information;
determining whether a dangerous object exists for each of the ground non-included area and the ground included area based on a result of the analysis;
operating an autonomous emergency braking (AEB) system of a vehicle when it is determined that the dangerous object exists; and
collecting speed information about a vehicle,
wherein the determining comprises determining whether the first object information included in the LiDAR data for the ground included area is related to a stopped object based on the speed information about the vehicle, and determining that a dangerous object exists in the ground included area when the first object information is related to a moving object.

### Claim 12 (depends on 11)

The method of claim 11, wherein the step of determining comprises determining that a dangerous object exists in the ground non-included area when the first object information is included in the LiDAR data for the ground non-included area.

### Claim 13 (depends on 11)

The method of claim 11, wherein the step of determining comprises determining that a dangerous object exists in the ground included area when an object related to the first object information and an object related to the second object information are the same based on the first information of the LiDAR data and the second object information of the image data.

### Claim 14 (depends on 11)

The method of claim 11, further comprising adjusting an object detection threshold value of the camera sensor based on a predetermined sensitivity adjustment condition.

### Claim 15 (depends on 11)

The method of claim 11, wherein:
the analyzing comprises extracting lane boundary information included in the image data; and
the determining comprises determining that a dangerous object exists in the ground included area when the first object information included in the LiDAR data for the ground included area is a stopped object existing out of a lane boundary.

### Claim 16

A method of operating an apparatus for detecting an object, comprising:
generating light distance and ranging LiDAR data by using a LiDAR sensor;
generating image data by using a camera sensor;
dividing a common detection area of the LiDAR sensor and the camera sensor into a ground non-included area and a ground included area;
analyzing at least one of the LiDAR data to extract first object information and the image data to extract second object information;
determining whether a dangerous object exists for each of the ground non-included area and the ground included area based on a result of the analysis;
operating an autonomous emergency braking (AEB) system of a vehicle when it is determined that the dangerous object exists; and
collecting speed information about a vehicle,
wherein the step of determining comprises determining that a dangerous object exists in the ground included area when the first object information about the ground included area is about a stopped object and the second object information about the stopped object is included in the image data.