# US 12,112,535 B2 · Systems and methods for effecting map layer updates based on collected sensor data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,535 B2
- Document ID: US12112535B2
- Status: Granted
- Kind: B2
- Application number: 16/916,015
- Filed: 2020-06-29
- Published: 2021-12-30
- Granted: 2024-10-08
- Earliest claimed: 2020-06-29
- Assignees: Lyft, Inc.
- Applicants: Lyft, Inc.
- Inventors: Kris Richard Efland; Nadha Nafeeza Gafoor; Nastaran Ghadar; Amruta Kiran Kulkarni; Meng Tao; Ziyi Zhao
- Web page: https://patentgrep.com/patent/US12112535B2

## CPC

- G01C 21/3859: {Differential updating map data}
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G01C 21/3822: {Road feature data, e.g. slope data}
- G01C 21/3841: {Data obtained from two or more sources, e.g. probe vehicles}
- G01C 21/3878: {Hierarchical structures, e.g. layering}
- G06V 10/803: {of input or preprocessed data}
- G06V 20/182: {Network patterns, e.g. roads or rivers}
- G06V 20/54: of traffic, e.g. cars on the road, trains or boats
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Examples disclosed herein may involve a computing system configured to (i) maintain a map that is representative of a real-world environment, the map including a plurality of layers that are each encoded with a different type of map data, (ii) obtain sensor data indicative of a given area of the real-world environment, (iii) based on an evaluation of the obtained sensor data and map data corresponding to the given area, detect that a change has occurred in the given area, (iv) based on the collected sensor data, derive information about the detected change including at least a type of the change and a location of the change, (v) based on the derived information about the detected change, determine that one or more layers of the map is impacted by the detected change, and (vi) effect an update to the one or more layers of the map based on the derived information about the change.

## Claims

### Claim 1

A computer-implemented method comprising:
maintaining a map that is representative of a real-world environment, the map comprising a plurality of layers, wherein each layer of the map is encoded with a different type of map data;
obtaining sensor data that is indicative of a given area of the real-world environment;
based on an evaluation of (i) the obtained sensor data and (ii) map data corresponding to the given area, detecting that a change has occurred in the given area involving an addition of a new semantic feature to the given area of the real-world environment;
based on the obtained sensor data, deriving information about the detected change to the given area, wherein the derived information about the detected change to the given area includes at least a type of the detected change and a location of the detected change;
based on the derived information about the detected change, determining that multiple different layers of the map are impacted by the detected change, wherein the multiple different layers comprise a semantic map layer that is encoded with semantic map data and at least one of a geometric map layer or a priors layer; and
effecting an update to the multiple different layers of the map based on the derived information about the detected change.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
after effecting the update to the multiple different layers of the map based on the derived information about the detected change, obtaining additional sensor data that indicates that the detected change has been removed from the given area; and
based on an evaluation of at least the additional obtained sensor data that indicates that the detected change has been removed from the given area, reverting the update to the multiple different layers of the map.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, further comprising:
after detecting that the detected change has occurred in the given area, requesting additional sensor data that is indicative of the given area; and
based on requesting the additional sensor data, obtaining the additional sensor data, wherein effecting the update to the multiple different layers of the map is based on the obtained additional sensor data.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein requesting additional sensor data comprises causing a sensor-equipped vehicle to traverse the given area, and wherein obtaining the additional sensor data comprises receiving the additional sensor data from the sensor-equipped vehicle.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the obtained sensor data comprises 2D image data and the map data corresponding to the given area comprises 2D reference images of the given area, and wherein the evaluation of (i) the obtained sensor data and (ii) the map data comprises:
localizing the 2D image data in relation to the 2D reference images; and
determining a delta between the 2D image data and the 2D reference images.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein obtaining sensor data that is indicative of the given area of the real-world environment comprises obtaining respective sensor data sets collected by two or more vehicles in the given area of the real-world environment, the method further comprising:
detecting the detected change in two or more of the respective sensor data sets;
based on detecting the detected change in two or more of the respective sensor data sets, determining a confidence level associated with the detected change; and
wherein effecting the update to the multiple different layers of the map is based on the determined confidence level exceeding a predetermined threshold.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein determining that the multiple different layers of the map are impacted by the detected change comprises:
determining a primary layer of the map to be updated based on the detected change; and
determining one or more additional layers of the map to which the update to the primary layer is to be propagated.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein detecting that the change has occurred in the given area comprises:
based on a snapping error that has occurred in the given area, detecting that the change has occurred in the given area.

### Claim 9

A non-transitory computer-readable medium comprising program instructions stored thereon that are executable to cause a computing system to:
maintain a map that is representative of a real-world environment, the map comprising a plurality of layers, wherein each layer of the map is encoded with a different type of map data;
obtain sensor data that is indicative of a given area of the real-world environment;
based on an evaluation of (i) the obtained sensor data and (ii) map data corresponding to the given area, detect that a change has occurred in the given area involving an addition of a new semantic feature to the given area of the real-world environment;
based on the obtained sensor data, derive information about the detected change to the given area, wherein the derived information about the detected change that has occurred in the given area includes at least a type of the detected change and a location of the detected change;
based on the derived information about the detected change, determine that multiple different layers of the map are impacted by the detected change, wherein the multiple different layers comprise a semantic map layer that is encoded with semantic map data and at least one of a geometric map layer or a priors layer; and
effect an update to the multiple different layers of the map based on the derived information about the detected change.

### Claim 10 (depends on 9)

The computer-readable medium of claim 9, wherein the computer-readable medium further comprises program instructions stored thereon that are executable to cause the computing system to:
after effecting the update to the multiple different layers of the map based on the derived information about the detected change, obtain additional sensor data that indicates that the detected change has been removed from the given area; and
based on an evaluation of at least the additional obtained sensor data that indicates that the detected change has been removed from the given area, revert the update to the multiple different layers of the map.

### Claim 11 (depends on 9)

The computer-readable medium of claim 9, wherein the computer-readable medium further comprises program instructions stored thereon that are executable to cause the computing system to:
after detecting that the detected change has occurred in the given area, request additional sensor data that is indicative of the given area; and
based on requesting the additional sensor data, obtain the additional sensor data, wherein effecting the update to the multiple different layers of the map is based on the obtained additional sensor data.

### Claim 12 (depends on 11)

The computer-readable medium of claim 11, wherein requesting additional sensor data comprises causing a sensor-equipped vehicle to traverse the given area, and wherein obtaining the additional sensor data comprises receiving the additional sensor data from the sensor-equipped vehicle.

### Claim 13 (depends on 9)

The computer-readable medium of claim 9, wherein the obtained sensor data comprises 2D image data and the map data corresponding to the given area comprises 2D reference images of the given area, and wherein the evaluation of (i) the obtained sensor data and (ii) the map data comprises:
localizing the 2D image data in relation to the 2D reference images; and
determining a delta between the 2D image data and the 2D reference images.

### Claim 14 (depends on 9)

The computer-readable medium of claim 9, wherein obtaining sensor data that is indicative of the given area of the real-world environment comprises obtaining respective sensor data sets collected by two or more vehicles in the given area of the real-world environment, and wherein the computer-readable medium further comprises program instructions stored thereon that are executable to cause the computing system to:
detect the detected change in two or more of the respective sensor data sets;
based on detecting the detected change in two or more of the respective sensor data sets, determine a confidence level associated with the detected change; and
wherein effecting the update to the multiple different layers of the map is based on the determined confidence level exceeding a predetermined threshold.

### Claim 15 (depends on 9)

The computer-readable medium of claim 9, wherein the program instructions that are executable to cause the computing system to determine that the multiple different layers of the map are impacted by the detected change comprise program instructions that are executable to cause the computing system to:
determine a primary layer of the map to be updated based on the detected change; and
determine one or more additional layers of the map to which the update to the primary layer is to be propagated.

### Claim 16 (depends on 9)

The computer-readable medium of claim 9, wherein the program instructions that are executable to cause the computing system to detect that the change has occurred in the given area comprise program instructions that are executable to cause the computing system to:
based on a snapping error that has occurred in the given area, detect that the change has occurred in the given area.

### Claim 17

A computing system comprising:
at least one processor;
a non-transitory computer-readable medium; and
program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor such that the computing system is capable of:
maintaining a map that is representative of a real-world environment, the map comprising a plurality of layers, wherein each layer of the map is encoded with a different type of map data;
obtaining sensor data that is indicative of a given area of the real-world environment;
based on an evaluation of (i) the obtained sensor data and (ii) map data corresponding to the given area, detecting that a change has occurred in the given area involving an addition of a new semantic feature to the given area of the real-world environment;
based on the obtained sensor data, deriving information about the detected change to the given area, wherein the derived information about the detected change to the given area includes at least a type of the detected change and a location of the detected change;
based on the derived information about the detected change, determining that multiple different layers of the map are impacted by the detected change, wherein the multiple different layers comprise a semantic map layer that is encoded with semantic map data and at least one of a geometric map layer or a priors layer; and
effecting an update to the multiple different layers of the map based on the derived information about the detected change.

### Claim 18 (depends on 17)

The computing system of claim 17, wherein the program instructions that are executable by the at least one processor such that the computing system is capable of determining that the multiple different layers of the map are impacted by the detected change comprise program instructions that are executable by the at least one processor such that the computing system is capable of:
determining a primary layer of the map to be updated based on the detected change; and
determining one or more additional layers of the map to which the update to the primary layer is to be propagated.