# US 2024/0402708 A1 · Autonomous driving vehicle and a method of driving the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0402708 A1
- Document ID: US20240402708A1
- Status: Published application
- Kind: A1
- Application number: 18/523,338
- Filed: 2023-11-29
- Published: 2024-12-05
- Earliest claimed: 2023-05-31
- Assignees: Hyundai Motor Company; Kia Corporation
- Applicants: Hyundai Motor Company; Kia Corporation
- Inventors: Jong Hyeok Park; Dong Hwan Kwak; Sang Hyeon Oh
- Web page: https://patentgrep.com/patent/US20240402708A1

## CPC

- G05D 1/0268: {using internal positioning means}
- B60W 50/0205: {Diagnosing or detecting failures; Failure detection models}
- B60W 50/023: Avoiding failures by using redundant parts
- B60W 50/029: Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts
- B60W 60/00186: {related to the vehicle}
- B60W 2050/0215: {Sensor drifts or sensor failures}
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2420/408: Radar; Laser, e.g. lidar
- B60W 2520/06: Direction of travel
- B60W 2520/10: Longitudinal speed
- B60W 2556/40: High definition maps
- G01C 21/1656: {with passive imaging devices, e.g. cameras}
- G01C 21/30: Map- or contour-matching
- G01C 21/3848: {Data obtained from both position sensors and additional sensors}
- G01S 7/497: Means for monitoring or calibrating
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/89: for mapping or imaging
- G01S 17/931: of land vehicles
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

An autonomous driving vehicle includes: a sensor section installed in the autonomous driving vehicle to sense direction data, position data, and velocity data of the autonomous driving vehicle; a camera configured to track movement of the autonomous driving vehicle and to estimate a position; a LiDAR sensor installed in the autonomous driving vehicle to generate LiDAR data; and a processor configured to receive the LiDAR data to generate a map. The processor stores image data for each predetermined driving distance using the camera. The processor also estimates a current position of the autonomous driving vehicle based on the image data stored in a predetermined range, based on a position where an abnormality occurs, and based on current image data captured while being driven in a failure mode in which the abnormality occurs in the LiDAR sensor.

## Claims

### Claim 1

An autonomous driving vehicle comprising:
at least one sensor installed in the autonomous driving vehicle to sense direction data, position data, and velocity data of the autonomous driving vehicle;
a camera configured to track movement of the autonomous driving vehicle and to estimate a position thereof;
a light detection and ranging (LiDAR) sensor installed in the autonomous driving vehicle to generate LiDAR data; and
a processor configured to receive the LiDAR data to generate a map, wherein the processor is configured to
store image data for each predetermined driving distance by use of the camera, and
estimate a current position of the autonomous driving vehicle based on the image data stored in a predetermined range, based on a position where an abnormality occurs in the LiDAR sensor, and based on current image data captured while the autonomous driving vehicle is driven in a failure mode in which the abnormality occurs in the LiDAR sensor.

### Claim 2 (depends on 1)

The autonomous driving vehicle according to claim 1, wherein, when the current position of the autonomous driving vehicle is estimated, the processor is further configured to guide the autonomous driving vehicle to a safe zone based thereon.

### Claim 3 (depends on 1)

The autonomous driving vehicle according to claim 1, wherein each of the stored image data and the current image data includes a position (odometry) of the autonomous driving vehicle, a direction (orientation) of the autonomous driving vehicle, and driving data with respect to time.

### Claim 4 (depends on 1)

The autonomous driving vehicle according to claim 1, wherein the processor is further configured to delete the stored image data based on importance of the stored image data.

### Claim 5 (depends on 4)

The autonomous driving vehicle according to claim 4, wherein the importance of the stored image data decreases as a storage time of the image data increases, or a position of the autonomous driving vehicle and a direction of the autonomous driving vehicle are included in a predetermined range and duplicated.

### Claim 6 (depends on 1)

The autonomous driving vehicle according to claim 1, wherein the processor is further configured to extract image feature points from the stored image data through an Oriented FAST and Rotated BRIEF (ORB) algorithm.

### Claim 7 (depends on 6)

The autonomous driving vehicle according to claim 6, wherein, when the abnormality occurs in the LiDAR sensor, the processor is further configured to estimate the current position of the autonomous driving vehicle by applying the stored image data and the current image data to a Local Bundle Adjustment (BA) algorithm.

### Claim 8 (depends on 1)

The autonomous driving vehicle according to claim 1, wherein the predetermined range includes image data stored at a position of a shortest distance based on the position where the abnormality occurs.

### Claim 9

A method of driving an autonomous driving vehicle including at least one sensor configured to sense direction data, position data, and velocity data of the autonomous driving vehicle, a camera configured to track movement of the autonomous driving vehicle and to estimate a position thereof, a light detection and ranging (LiDAR) sensor to generate LiDAR data, and a processor configured to receive the LiDAR data to generate a map, under control of the processor, the method comprising:
storing image data for each predetermined driving distance by use of the camera; and
estimating a current position of the autonomous driving vehicle based on the image data stored in a predetermined range, based on a position where an abnormality occurs in the LiDAR sensor, and based on current image data captured while the autonomous driving vehicle is driven in a failure mode in which the abnormality occurs.

### Claim 10 (depends on 9)

The method according to claim 9, further comprising, when the current position of the autonomous driving vehicle is estimated, guiding the autonomous driving vehicle to a safe zone based thereon under control of the processor.

### Claim 11 (depends on 9)

The method according to claim 9, wherein each of the stored image data and the current image data includes a position of the autonomous driving vehicle, a direction of the autonomous driving vehicle, and driving data with respect to time.

### Claim 12 (depends on 9)

The method according to claim 9, wherein the stored image data is deleted based on importance under control of the processor.

### Claim 13 (depends on 12)

The method according to claim 12, wherein the importance of the stored image data decreases as a storage time of the image data increases, or a position of the autonomous driving vehicle and a direction of the autonomous driving vehicle are included in a predetermined range and duplicated.

### Claim 14 (depends on 9)

The method according to claim 9, further comprising extracting image feature points from the stored image data through an Oriented FAST and Rotated BRIEF (ORB) algorithm under control of the processor.

### Claim 15 (depends on 14)

The method according to claim 14, wherein, when the abnormality occurs in the LiDAR sensor, estimating the current position of the autonomous driving vehicle comprises estimating the current position by applying the stored image data and the current image data to a Local Bundle Adjustment (BA) algorithm under control of the processor.

### Claim 16 (depends on 9)

The method according to claim 9, wherein the predetermined range includes image data stored at a position of a shortest distance based on the position where the abnormality occurs.
