# US 2021/0278221 A1 · Lane marking localization and fusion

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0278221 A1
- Document ID: US20210278221A1
- Status: Published application
- Kind: A1
- Application number: 17/320,888
- Filed: 2021-05-14
- Published: 2021-09-09
- Earliest claimed: 2018-02-14
- Assignees: Tusimple, Inc.
- Applicants: Tusimple, Inc.
- Inventors: Mingdong Wang; Chenzhe Qian; Xue Mei
- Web page: https://patentgrep.com/patent/US20210278221A1

## CPC

- G01C 21/30: Map- or contour-matching
- G01C 21/3658: {Lane guidance}
- G01C 21/3602: {Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera}
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Various embodiments provide a system and method for iterative lane marking localization that may be utilized by autonomous or semi-autonomous vehicles traveling within the lane. In an embodiment, the system comprises a locating device adapted to determine the vehicle's geographic location; a database; a region map; a response map; a plurality of cameras; and a computer connected to the locating device, database, and cameras, wherein the computer is adapted to receive the region map, wherein the region map corresponds to a specified geographic location; generate the response map by receiving information from the camera, the information relating to the environment in which the vehicle is located; identifying lane markers observed by the camera; and plotting identified lane markers on the response map; compare the response map to the region map; and iteratively generate a predicted vehicle location based on the comparison of the response map and the region map.

## Claims

### Claim 1

A method for determining a vehicle's location, comprising:
approximating a vehicle's region;
receiving a first region map and a second region map from a database, wherein each of the first region map and the second region map corresponds to the vehicle's approximated region;
receiving a first response image and a second response image generated by different imaging devices respectively, the first response image and the second response image comprising information relating to the vehicle's environment;
generating a first response map from the first response image, and a second response map from the second response image, wherein each of the first response map and the second response map corresponds to the vehicle's location;
generating a first corrected location based on comparing the first response map to the first region map;
generating a second corrected location based on comparing the second response map to the second region map; and
predicting the vehicle's location based on the first corrected location and the second corrected location.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first region map comprises a plurality of first region points indicating an expected roadway lane, the wherein the second region map comprises a plurality of second region points indicating the expected roadway lane.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
receiving a third region map from the database, wherein the third region map corresponds to the vehicle's approximated region;
receiving a third response image generated by an imaging device, the third response image comprising information relating to the vehicle's environment;
generating a third response map from the third response image, the third response map corresponding to the vehicle's location; and
generating a third corrected location based on comparing the third response map to the third region map,
wherein predicting the vehicle's location is further based on the third corrected location.

### Claim 4 (depends on 3)

The method of claim 3, wherein the first response image is generated by a first imaging device, the second response image is generated by a second imaging device, and the third response image is generated by a third imaging device, wherein the first imaging device, the second imaging device, and the third imaging device are each adapted to perceive different aspects of the vehicle's environment.

### Claim 5 (depends on 1)

The method of claim 1, wherein each of the first region map and the second region map comprises information from a previously generated vehicle location.

### Claim 6

A system for determining a vehicle's location, comprising:
a data processor; and
a memory storing a computer module, executable by the data processor to:
approximate a vehicle's region;
receive a first region map and a second region map from a database, wherein each of the first region map and the second region map corresponds to the vehicle's approximated region;
receive a first response image and a second response image generated by different imaging devices respectively, the first response image and the second response image comprising information relating to the vehicle's environment;
generate a first response map from the first response image, and a second response map from the second response image, wherein each of the first response map and the second response map corresponds to the vehicle's location;
generate a first corrected location based on comparing the first response map to the first region map;
generate a second corrected location based on comparing the second response map to the second region map; and
predict the vehicle's location based on the first corrected location and the second corrected location.

### Claim 7 (depends on 6)

The system of claim 6, wherein the first response map comprises a plurality of first response points indicating the vehicle's location.

### Claim 8 (depends on 7)

The system of claim 7, wherein the memory storing the computer module, is executable by the data processor further to:
detect lane markers in the first response image, the lane markers pertaining to physical aspects contained in the first response image; and
plot the plurality of first response points on the first response map, the plurality of first response points indicating locations of the lane markers.

### Claim 9 (depends on 6)

The system of claim 6, wherein the memory storing the computer module, is executable by the data processor further to:
generate a first confidence score based on comparing the first response map to the first region map.

### Claim 10

A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
approximate a vehicle's region;
receive a first region map and a second region map from a database, wherein each of the first region map and the second region map corresponds to the vehicle's approximated region;
receive a first response image and a second response image generated by different imaging devices respectively, the first response image and the second response image comprising information relating to the vehicle's environment;
generate a first response map from the first response image, and a second response map from the second response image, wherein each of the first response map and the second response map corresponds to the vehicle's location;
generate a first corrected location based on comparing the first response map to the first region map;
generate a second corrected location based on comparing the second response map to the second region map; and
predict the vehicle's location based on the first corrected location and the second corrected location.

### Claim 11 (depends on 10)

The non-transitory machine-useable storage medium of claim 10, wherein each of the first response image and the second response image is generated from radar sensing equipment, light detection and ranging (LIDAR) sensing equipment, Global Positioning System (GPS) sensing information, and/or images.

### Claim 12 (depends on 10)

The non-transitory machine-useable storage medium of claim 10, when executed by the machine, further cause the machine to:
output the vehicle's predicted location to an advanced driver-assistance system (ADAS).

### Claim 13

A method for determining a vehicle's location, comprising:
receiving, from a locating device adapted to determine a vehicle's geographic region, the vehicle's predicted geographic region;
receiving, from a database, a first region map and a second region map, wherein each of the first region map and the second region map corresponds to the vehicle's predicted geographic region;
receiving, from different imaging devices respectively, a first perceived information and a second perceived information, wherein the first perceived information and the second perceived information relate to the vehicle's environment;
generating a first response map based on the first perceived information, and a second response map based on the second perceived information, wherein the first response map corresponds to lane markers detected within the first response map, and wherein the second response map corresponds to lane markers detected within the second response map;
generating a first corrected location based on comparing the first response map to the first region map;
generating a second corrected location based on comparing the second response map to the second region map; and
determining a fused vehicle location based on the first corrected location and the second corrected location.

### Claim 14 (depends on 13)

The method of claim 13, wherein the fused vehicle location is further based on a previously determined vehicle location.

### Claim 15

A system for determining a vehicle's location, comprising:
a processor configured to:
receive, from a locating device adapted to determine a vehicle's geographic region, the vehicle's predicted geographic region;
receive, from a database, a first region map and a second region map, wherein each of the first region map and the second region map corresponds to the vehicle's predicted geographic region;
receive, from different imaging devices respectively, a first perceived information and a second perceived information, wherein the first perceived information and the second perceived information relate to the vehicle's environment;
generate a first response map based on the first perceived information, and a second response map based on the second perceived information, wherein the first response map corresponds to lane markers detected within the first response map, and wherein the second response map corresponds to lane markers detected within the second response map;
generate a first corrected location based on comparing the first response map to the first region map;
generate a second corrected location based on comparing the second response map to the second region map; and
determine a fused vehicle location based on the first corrected location and the second corrected location.

### Claim 16 (depends on 15)

The system of claim 15, wherein the processor is further configured to:
receive, from the database, a third region map corresponding to the vehicle's predicted geographic region;
receive, from an imaging device, a third perceived information relating to the vehicle's environment;
generate a third response map based on the third perceived information, wherein the third response map corresponds to lane markers detected within the third response map;
generate a third corrected location based on comparing the third response map to the third region map,
wherein the fused vehicle location is further based on the third corrected location.

### Claim 17 (depends on 15)

The system of claim 15, wherein the locating device comprises a Global Positioning System (GPS) device or an inertial measurement unit (IMU).

### Claim 18 (depends on 15)

The system of claim 15, wherein the imaging devices comprise a camera or a light detection and ranging (LIDAR) device.

### Claim 19

A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:
receive, from a locating device adapted to determine a vehicle's geographic region, the vehicle's predicted geographic region;
receive, from a database, a first region map and a second region map, wherein each of the first region map and the second region map corresponds to the vehicle's predicted geographic region;
receive, from different imaging devices respectively, a first perceived information and a second perceived information, wherein the first perceived information and the second perceived information relate to the vehicle's environment;
generate a first response map based on the first perceived information, and a second response map based on the second perceived information, wherein the first response map corresponds to lane markers detected within the first response map, and wherein the second response map corresponds to lane markers detected within the second response map;
generate a first corrected location based on comparing the first response map to the first region map;
generate a second corrected location based on comparing the second response map to the second region map; and
determine a fused vehicle location based on the first corrected location and the second corrected location.

### Claim 20 (depends on 19)

The non-transitory machine-useable storage medium of claim 19, when executed by the machine, further cause the machine to determine a confidence score, wherein the confidence score comprises a variance that is computed using a logistic function.
