# US 12,007,784 B2 · Method and apparatus for self localization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,007,784 B2
- Document ID: US12007784B2
- Status: Granted
- Kind: B2
- Application number: 16/830,772
- Filed: 2020-03-26
- Published: 2021-09-30
- Granted: 2024-06-11
- Earliest claimed: 2020-03-26
- Assignees: HERE Global B.V.
- Applicants: HERE Global B.V.
- Inventors: Andi Zang; Xin Chen
- Web page: https://patentgrep.com/patent/US12007784B2

## CPC

- G01C 21/30: Map- or contour-matching
- G05D 1/0268: {using internal positioning means}
- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- G05D 1/0248: {in combination with a laser (lasers per se H01S)}
- G05D 1/0278: {using satellite positioning signals, e.g. GPS}
- G05D 1/245: using dead reckoning
- G05D 1/247: using signals provided by artificial sources external to the vehicle, e.g. navigation beacons
- G05D 1/248: generated by satellites, e.g. GPS
- H04W 4/029: Location-based management or tracking services
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Methods described herein relate to self-localization. Methods may include: receiving sensor data from a vehicle traveling along a road segment; identifying one or more objects of the environment from the sensor data; determining a coarse location of the vehicle; comparing the identified one or more objects with corresponding ground truth objects; calculating an observation in response to the comparison of the identified one or more objects with the corresponding ground truth objects; determining a location of the vehicle with a higher precision than the coarse location based, at least in part, on the observation; and allocating resources of the apparatus based, at least in part, on the observation.

## Claims

### Claim 1

An apparatus comprising at least one processor and at least one non-transitory memory including computer program code instructions, the computer program code instructions configured to, when executed, cause the apparatus to at least:
receive sensor data from a vehicle traveling along a road segment, wherein the sensor data comprises first sensor data of discrete objects in an environment from a first sensor and second sensor data of continuous objects in the environment from a second sensor;
determine a coarse location of the vehicle;
identify one or more objects of the environment from the sensor data, wherein the one or more objects of the environment from the sensor data comprise at least one of one or more discrete objects from the first sensor data or one or more continuous objects from the second sensor data;
compare the identified one or more discrete objects from the first sensor data and the identified one or more continuous objects from the second sensor data with corresponding ground truth objects;
calculate an observation in response to the comparison of the identified one or more objects with the corresponding ground truth objects, wherein the calculated observation includes an accuracy of localization relative to the first sensor data of discrete objects in the environment and the second sensor data of continuous objects in the environment, wherein the calculated observation further includes an indication that the first sensor data has a relatively higher quality of object matching than the second sensor data;
determine a location of the vehicle with a higher precision than the coarse location based, at least in part, on the calculated observation; and
allocate resources of the apparatus based, at least in part, on the calculated observation, wherein causing the apparatus to allocate resources comprises causing the apparatus to prioritize the first sensor data from the first sensor over the second sensor data from the second sensor based, at least in part, on the calculated observation.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein causing the apparatus to identify one or more objects of the environment from the sensor data comprises causing the apparatus to:
using a first algorithm of a first module, identify one or more pole objects in the environment from the first sensor data; and
using a second algorithm of a second module, identify one or more lane line objects in the environment from the second sensor data.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein causing the apparatus to compare the identified one or more pole objects from the first sensor data and the identified one or more lane line objects from the second sensor data with corresponding ground truth objects comprises causing the apparatus to:
compare the identified one or more pole objects in the environment with corresponding ground truth pole objects; and
compare the identified one or more lane line objects in the environment with corresponding ground truth lane line objects.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein causing the apparatus to calculate the observation in response to the comparison of the identified one or more pole objects from the first sensor data and the identified one or more lane line objects from the second sensor data with the corresponding ground truth objects comprises causing the apparatus to:
calculate a first observation in response to the comparison of the identified one or more pole objects in the environment with the corresponding ground truth pole objects; and
calculate a second observation in response to the comparison of the identified one or more lane line objects in the environment with corresponding ground truth lane line objects.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein causing the apparatus to allocate resources of the apparatus based, at least in part, on the observation further comprises causing the apparatus to:
analyze resource consumption of the first module and the second module; and
allocate proportionally more resources to at least one of the first module or the second module in response to the analysis of resource consumption.

### Claim 6 (depends on 4)

The apparatus of claim 4, wherein causing the apparatus to allocate resources of the apparatus based, at least in part, on the observation comprises causing the apparatus to:
identify a model based, at least in part, on the calculated first observation and the calculated second observation; and
allocate resources according to the identified model.

### Claim 7 (depends on 2)

The apparatus of claim 2, wherein the apparatus is further caused to:
receive sensor data from at least one of a global positioning system (GPS) or inertial measurement unit (IMU); and
identify, at a third module, the coarse location based on the sensor data from the at least one of the GPS or IMU;
wherein causing the apparatus to allocate resources of the apparatus based, at least in part, on the observation further comprises causing the apparatus to allocate proportionally more resources to the first module, the second module, or the third module based, at least in part, on the observation and the coarse location identified at the third module.

### Claim 8

A computer program product comprising at least one non-transitory computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to:
receive sensor data from a vehicle traveling along a road segment, wherein the sensor data comprises first sensor data of one or more discrete objects in an environment from a first sensor and second sensor data of one or more continuous objects in the environment from a second sensor;
determine a coarse location for the vehicle;
identify one or more objects of the environment from the sensor data, wherein the one or more objects of the environment from the sensor data comprises at least one of one or more discrete objects from the first sensor data or one or more continuous objects from the second sensor data;
compare the identified one or more discrete objects from the first sensor data and the identified one or more continuous objects from the second sensor data with corresponding ground truth objects;
calculate an observation in response to the comparison of the identified one or more objects with the corresponding ground truth objects, wherein the calculated observation includes an accuracy of localization relative to the first sensor data of discrete objects in the environment and the second sensor data of continuous objects in the environment, wherein the calculated observation further includes an indication that the first sensor data has a relatively higher quality of object matching than the second sensor data;
determine a location of the vehicle with a higher precision than the coarse location based, at least in part, on the calculated observation; and
allocate resources of the apparatus based, at least in part, on the calculated observation, wherein causing the apparatus to allocate resources comprises causing the apparatus to prioritize the first sensor data from the first sensor over the second sensor data from the second sensor based, at least in part, on the calculated observation.

### Claim 9 (depends on 8)

The computer program product of claim 8, wherein the program code instructions to identify one or more objects of the environment from the sensor data comprise program code instructions to:
using a first algorithm of a first module, identify one or more pole objects in the environment from the first sensor data; and
using a second algorithm of a second module, identify one or more lane line objects in the environment from the second sensor data.

### Claim 10 (depends on 9)

The computer program product of claim 9, wherein the program code instructions to compare the identified one or more pole objects from the first sensor data and the identified one or more lane line objects from the second sensor data with corresponding ground truth objects comprise program code instructions to:
compare the identified one or more pole objects in the environment with corresponding ground truth pole objects; and
compare the identified one or more lane line objects in the environment with corresponding ground truth lane line objects.

### Claim 11 (depends on 10)

The computer program product of claim 10, wherein the program code instructions to calculate the observation in response to the comparison of the identified one or more pole objects from the first sensor data and the identified one or more lane line objects from the second sensor data with the corresponding ground truth object comprises program code instructions to:
calculate a first observation in response to the comparison of the one or more pole objects in the environment with the corresponding ground truth pole objects; and
calculate a second observation in response to the comparison of the one or more lane line objects in the environment with corresponding ground truth lane line objects.

### Claim 12 (depends on 11)

The computer program product of claim 11, wherein the program code instructions to allocate resources of the apparatus based, at least in part, on the observation further comprises program code instructions to:
analyze resource consumption of the first module and the second module; and
allocate proportionally more resources to at least one of the first module or the second module in response to the analysis of resource consumption.

### Claim 13 (depends on 11)

The computer program product of claim 11, wherein the program code instructions to allocate resources of the apparatus based, at least in part, on the observation comprise program code instructions to:
identify a model based, at least in part, on the calculated first observation and the calculated second observation; and
allocate resources according to the identified model.

### Claim 14 (depends on 9)

The computer program product of claim 9, further comprising program code instructions to:
receive sensor data from at least one of a global positioning system (GPS) or inertial measurement unit (IMU); and
identify, at a third module, the coarse location based on the sensor data from the at least one of the GPS or IMU;
wherein the program code instructions to allocate resources of the apparatus based, at least in part, on the observation further comprises program code instructions to allocate proportionally more resources to the first module, the second module, or the third module based, at least in part, on the observation and the coarse location identified at the third module.

### Claim 15

A method comprising:
receiving sensor data from a vehicle traveling along a road segment, wherein the sensor data comprises first sensor data of one or more discrete objects in an environment from a first sensor and second sensor data of one or more continuous objects in the environment from a second sensor;
identifying one or more objects of the environment from the sensor data, wherein the one or more objects of the environment from the sensor data comprises at least one of one or more discrete objects from the first sensor data or one or more continuous objects from the second sensor data;
determining a coarse location of the vehicle;
comparing the identified one or more discrete objects from the first sensor data and the identified one or more continuous objects from the second sensor data with corresponding ground truth objects;
calculating an observation in response to the comparison of the identified one or more objects with the corresponding ground truth objects, wherein the calculated observation includes an accuracy of localization relative to the first sensor data of discrete objects in the environment and the second sensor data of continuous objects in the environment, wherein the calculated observation further includes an indication that the first sensor data has a relatively higher quality of object matching than the second sensor data;
determining a location of the vehicle with a higher precision than the coarse location based, at least in part, on the calculated observation; and
allocating resources of the apparatus based, at least in part, on the calculated observation, wherein causing the apparatus to allocate resources comprises causing the apparatus to prioritize the first sensor data from the first sensor over the second sensor data from the second sensor based, at least in part, on the calculated observation.

### Claim 16 (depends on 15)

The method of claim 15, wherein identifying one or more objects of the environment from the sensor data comprises:
using a first algorithm of a first module, identifying one or more pole objects in the environment from the first sensor data; and
using a second algorithm of a second module, identifying one or more lane line objects in the environment from the second sensor data.

### Claim 17 (depends on 16)

The method of claim 16, wherein comparing the identified one or more pole objects from the first sensor data and the identified one or more lane line objects from the second sensor data with corresponding ground truth objects comprises:
comparing the identified one or more pole objects in the environment with corresponding ground truth pole objects; and
comparing the identified one or more lane line objects in the environment with corresponding ground truth lane line objects.

### Claim 18 (depends on 17)

The method of claim 17, wherein calculating the observation in response to the comparison of the identified one or more pole objects from the first sensor data and the identified one or more lane line objects from the second sensor data with the corresponding ground truth objects comprises:
calculating a first observation in response to the comparison of the one or more pole objects in the environment with the corresponding ground truth pole objects; and
calculating a second observation in response to the comparison of the one or more lane line objects in the environment with corresponding ground truth lane line objects.

### Claim 19 (depends on 1)

The apparatus of claim 1, wherein causing the apparatus to allocate resources comprises causing the apparatus to allocate relatively more of at least one of processor capacity, memory capacity, or communications bandwidth to the first sensor data from the first sensor over the second sensor data from the second sensor based, at least in part, on the observation.

### Claim 20 (depends on 1)

The apparatus of claim 1, wherein the first sensor is a first sensor type selected from the group of an image sensor and a Light Detection and Ranging (LiDAR) sensor, wherein the second sensor is a second sensor type selected from the group of an image sensor and a Light Detection and Ranging (LiDAR) sensor, and wherein the first sensor type is different from the second sensor type.