# US 2021/0278232 A1 · Lane marking localization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0278232 A1
- Document ID: US20210278232A1
- Status: Published application
- Kind: A1
- Application number: 17/308,803
- Filed: 2021-05-05
- Published: 2021-09-09
- Earliest claimed: 2018-02-14
- Assignees: Tusimple, Inc.
- Applicants: Tusimple, Inc.
- Inventors: Mingdong WANG; Chenzhe QIAN; Xue MEI
- Web page: https://patentgrep.com/patent/US20210278232A1

## CPC

- G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera}
- G08G 1/137: the indicator being in the form of a map
- G01C 21/32: Structuring or formatting of map data
- G08G 1/133: within the vehicle {; Indicators inside the vehicles or at stops}
- G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection}
- G01S 13/89: for mapping or imaging
- G01S 17/89: for mapping or imaging
- G01S 19/13: Receivers
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

Various embodiments of the present disclosure provide a system and method for lane marking localization that may be utilized by autonomous or semi-autonomous vehicles traveling within the lane. In the embodiment, the system comprises a locating device adapted to determine the vehicle's geographic location; a database; a region map; a response map; a camera; and a computer connected to the locating device, database, and camera, wherein the computer is adapted to: receive the region map, wherein the region map corresponds to a specified geographic location; generate the response map by receiving information form the camera, the information relating to the environment in which the vehicle is located; identifying lane markers observed by the camera; and plotting identified lane markers on the response map; compare the response map to the region map; and generate a predicted vehicle location based on the comparison of the response map and region map.

## Claims

### Claim 1

A method implemented by a processor, comprising:
approximating a vehicle's region;
receiving a region map from a database, wherein the region map corresponds to the vehicle's approximated region and comprises a plurality of region points indicating an expected roadway lane;
receiving a response image generated by an imaging device of one or more imaging devices of the vehicle, the response image comprising information relating to the vehicle's environment;
generating a response map from the response image, the response map comprising a plurality of response points indicating the vehicle's location;
comparing the response map to the region map to determine (a) differences between the plurality of response points and the plurality of region points, and (b) a confidence score that comprises a variance that is computed using a logistic function; and
predicting the vehicle's roadway position based on the confidence score and the differences between the plurality of response points and the plurality of region points.

### Claim 2 (depends on 1)

The method of claim 1, wherein the vehicle's region is approximated using a Global Positioning System (GPS) device or an inertial measurement unit (IMU) device.

### Claim 3 (depends on 1)

The method of claim 1, wherein the generating the response map further comprises:
detecting lane markers in the response image, the lane markers pertaining to physical aspects contained in the response image; and
plotting the response points on the response map, the response points indicating locations of the lane markers.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
generating, based on the differences, the confidence score.

### Claim 5 (depends on 1)

The method of claim 1, wherein the response image is generated from radar sensing equipment, light detection and ranging (LIDAR) sensing equipment, Global Positioning System (GPS) sensing information, and/or images.

### Claim 6 (depends on 1)

The method of claim 1, wherein the region map and the response map are compared at a selected frequency.

### Claim 7 (depends on 6)

The method of claim 6, wherein the selected frequency is at least 20 cycles per second.

### Claim 8

An apparatus, comprising:
a processor configured to:
approximate a vehicle's region;
receive a region map from a database, wherein the region map corresponds to the vehicle's approximated region and comprises a plurality of region points indicating an expected roadway lane;
receive a response image generated by an imaging device of one or more imaging devices of the vehicle, the response image comprising information relating to the vehicle's environment;
generate a response map from the response image, the response map comprising a plurality of response points indicating the vehicle's location;
compare the response map to the region map to determine (a) differences between the plurality of response points and the plurality of region points, and (b) a confidence score that comprises a variance that is computed using a logistic function; and
predict the vehicle's roadway position based on the confidence score and the differences between the plurality of response points and the plurality of region points.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the processor is further configured to:
output the vehicle's predicted location to an advanced driver-assistance system (ADAS).

### Claim 10 (depends on 8)

The apparatus of claim 8, wherein each of the one or more imaging devices is adapted to perceive a different aspect of the vehicle's environment.

### Claim 11 (depends on 8)

The apparatus of claim 8, wherein the variance is computed as:
variance
=
x
min
+
(
x
max
-
x
min
)
1
+
e
S
⁡
(
G
⁢
x
-
m
)
,
wherein x is a matching score of the response map, xmin is a minimum value of the response map, xmax is a maximum value of the response map, S is a steepness parameter, G is a growth rate parameter, and m is a midpoint of the response map.

### Claim 12 (depends on 8)

The apparatus of claim 8, wherein the imaging device comprises a light detection and ranging (LIDAR) device.

### Claim 13 (depends on 8)

The apparatus of claim 8, wherein the imaging device comprises a camera that captures images at a rate of at least 10 images per second.

### Claim 14

A non-transitory computer-readable storage medium having instructions stored thereupon, the storage medium comprising:
instructions for approximating a vehicle's region;
instructions for receiving a region map from a database, wherein the region map corresponds to the vehicle's approximated region and comprises a plurality of region points indicating an expected roadway lane;
instructions for receiving a response image generated by an imaging device of one or more imaging devices of the vehicle, the response image comprising information relating to the vehicle's environment;
instructions for generating a response map from the response image, the response map comprising a plurality of response points indicating the vehicle's location;
instructions for comparing the response map to the region map to determine (a) differences between the plurality of response points and the plurality of region points, and (b) a confidence score that comprises a variance that is computed using a logistic function; and
instructions for predicting the vehicle's roadway position based on the confidence score and the differences between the plurality of response points and the plurality of region points.

### Claim 15 (depends on 14)

The storage medium of claim 14, further comprising:
instructions for comparing the confidence score to a threshold; and
instructions for outputting the vehicle's predicted roadway position based on the confidence score being lower than the threshold.

### Claim 16 (depends on 14)

The storage medium of claim 14, wherein the response map is generated using a camera pointing in a direction opposite to a direction of travel of the vehicle.

### Claim 17 (depends on 14)

The storage medium of claim 14, wherein lane markers in the response map are based on one or more physical structures.

### Claim 18 (depends on 17)

The storage medium of claim 17, wherein the one or more physical structures comprise a bridge, a barrier, a sign, or a building.

### Claim 19 (depends on 17)

The storage medium of claim 17, wherein the lane markers comprise at least one of solid lines, striped lines, or reflectors.

### Claim 20 (depends on 14)

The storage medium of claim 14, wherein the region map further comprises information related to a speed of the collection vehicle, a direction of travel of the collection vehicle, or a velocity of the collection vehicle relative to the expected roadway lane.
