# US 2022/0036069 A1 · Detecting road edges by fusing aerial image and telemetry evidences

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0036069 A1
- Document ID: US20220036069A1
- Status: Published application
- Kind: A1
- Application number: 16/943,673
- Filed: 2020-07-30
- Published: 2022-02-03
- Earliest claimed: 2020-07-30
- Assignees: GM Global Technology Operations LLC
- Applicants: GM Global Technology Operations LLC
- Inventors: Orhan Bulan; Yehenew G. Mengistu; Sheetal Mahesh; David H. Clifford
- Web page: https://patentgrep.com/patent/US20220036069A1

## CPC

- G06V 20/182: {Network patterns, e.g. roads or rivers}
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06T 7/13: Edge detection
- G06T 7/143: involving probabilistic approaches, e.g. Markov random field [MRF] modelling
- G06T 7/174: involving the use of two or more images
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/809: {of classification results, e.g. where the classifiers operate on the same input data}
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30256: Lane; Road marking
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method to detect a roadway edge includes calculating a first likelihood of a roadway edge from an aerial image of a roadway by shifting a centerline of the roadway perpendicular to the centerline and overlapping the centerline with image gradients. A second likelihood of the roadway edge is determined using a vehicle telemetry fitting a probability distribution to telemetry points along the roadway. The first likelihood of the roadway edge and the second likelihood of the roadway edge are fused to identify a final likelihood of the roadway edge.

## Claims

### Claim 1

A system to detect a roadway edge, comprising:
a first likelihood of a roadway edge calculated from an aerial image of a roadway;
a second likelihood of the roadway edge determined using a vehicle telemetry and a probability distribution fitted to telemetry points along the roadway; and
a fusion of the first likelihood of the roadway edge and the second likelihood of the roadway edge used to identify a final likelihood of the roadway edge.

### Claim 2 (depends on 1)

The system to detect a roadway edge of claim 1, including multiple data points representative of the telemetry points collected over time from multiple vehicles traveling on the roadway.

### Claim 3 (depends on 2)

The system to detect a roadway edge of claim 2, including the multiple data points being configured in multiple groups including a first group representing vehicle travel in a first travel direction.

### Claim 4 (depends on 3)

The system to detect a roadway edge of claim 3, including the multiple data points being configured in multiple groups including a second group representing vehicle travel in a second travel direction opposite to the first travel direction.

### Claim 5 (depends on 4)

The system to detect a roadway edge of claim 4, wherein:
the roadway includes a centerline, the first likelihood of the roadway edge calculated by shifting the centerline perpendicular to the centerline and overlapping the aerial image with image gradients; and
the first likelihood of the roadway edge further determined using at least one of: a 95th percentile of the data points in the first travel direction determined with respect to the centerline of the first group of the data points; based on a type of the roadway; and a reported quantity of roadway lanes.

### Claim 6 (depends on 5)

The system to detect a roadway edge of claim 5, wherein the second likelihood of the roadway edge is further determined using at least one of: a 95th percentile of the data points in the second travel direction determined with respect to the centerline of the second group of the data points; based on the type of the roadway; and the reported quantity of roadway lanes.

### Claim 7 (depends on 1)

The system to detect a roadway edge of claim 1, further including an aerial image data map generated using aerial image data of the roadway.

### Claim 8 (depends on 7)

The system to detect a roadway edge of claim 7, further including multiple data points including at least one object defining the roadway edge.

### Claim 9 (depends on 8)

The system to detect a roadway edge of claim 8, further including a graph presenting the first likelihood using geolocations identifying the roadway edge.

### Claim 10 (depends on 9)

The system to detect a roadway edge of claim 9, wherein the graph presents at least one of: a 95th percentile probability of a likelihood of the geolocations of the at least one object, a type of the roadway; and a reported quantity of roadway lanes.

### Claim 11

A method to detect a roadway edge, comprising:
calculating a first likelihood of a roadway edge from an aerial image of a roadway by shifting a centerline of the roadway perpendicular to the centerline and overlapping the centerline with image gradients;
determining a second likelihood of the roadway edge using a vehicle telemetry fitting a probability distribution to telemetry points along the roadway; and
fusing the first likelihood of the roadway edge and the second likelihood of the roadway edge to identify a final likelihood of the roadway edge.

### Claim 12 (depends on 11)

The method to detect a roadway edge of claim 11, further comprising identifying multiple data points representative of the telemetry points collected over time from multiple vehicles traveling on the roadway.

### Claim 13 (depends on 12)

The method to detect a roadway edge of claim 12, further comprising grouping the data points into multiple groups including a first group representing vehicle travel in a first travel direction, and in a second group representing vehicle travel in a second travel direction opposite to the first travel direction.

### Claim 14 (depends on 13)

The method to detect a roadway edge of claim 13, further comprising determining the second likelihood of the roadway edge by calculating a 95th percentile of the data points in the first travel direction with respect to the centerline of the first group of the data points.

### Claim 15 (depends on 14)

The method to detect a roadway edge of claim 14, further comprising determining the second likelihood of the roadway edge by calculating a 95th percentile of the data points in the second travel direction with respect to the centerline of the second group of the data points.

### Claim 16 (depends on 11)

The method to detect a roadway edge of claim 11, further comprising:
generating an aerial image data map using aerial image data of an area of the roadway; and
removing extraneous data from the aerial image data map.

### Claim 17 (depends on 16)

The method to detect a roadway edge of claim 16, further comprising:
identifying data points including at least one object defining the roadway edge; and
generating a graph presenting the first likelihood using geolocations identifying the roadway edge by presenting 95th percentile probabilities of a likelihood of the geolocations of the at least one object.

### Claim 18

A method to detect a roadway edge, comprising:
identifying a centerline of a roadway;
calculating a first likelihood of a roadway edge from an aerial image of the roadway by shifting the centerline of the roadway perpendicular to the centerline;
determining a second likelihood of the roadway edge using a vehicle telemetry fitting a probability distribution to telemetry points of vehicle telemetry having a first group representing vehicle travel in a first travel direction and a second group representing vehicle travel in a second travel direction opposite to the first travel direction along the roadway; and
fusing a 95th percentile of the first likelihood of the roadway edge and a 95th percentile of the second likelihood of the roadway edge to identify a final likelihood of the roadway edge.

### Claim 19 (depends on 18)

The method to detect a roadway edge of claim 18, further including shifting the centerline in predetermined increments of a distance from the centerline.

### Claim 20 (depends on 18)

The method to detect a roadway edge of claim 18, further including overlapping the centerline with image gradients during calculation of the first likelihood of the roadway edge.