# US 11,443,530 B2 · Method for lane and roadway detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,443,530 B2
- Document ID: US11443530B2
- Status: Granted
- Kind: B2
- Application number: 17/053,882
- Filed: 2019-05-06
- Published: 2021-07-29
- Granted: 2022-09-13
- Earliest claimed: 2018-05-09
- Assignees: Conti Temic microelectronic GmbH
- Applicants: Conti Temic microelectronic GmbH
- Inventors: Florian Damerow; Andreea Mirela Fischer; Christopher Linsenmann; Kai Ide; Matthias Limmer; Stefan Hegemann
- Web page: https://patentgrep.com/patent/US11443530B2

## CPC

- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06T 7/12: Edge-based segmentation
- G06V 10/82: using neural networks
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method for lane and roadway detection uses a multitask DNN architecture including an encoder, a first decoder and a second decoder. The method includes the following steps: providing an input image by an optical detection device, filtering the input image by the encoder, generating a first representation of the lane and/or roadway by the encoder, processing the first representation in the first and second decoders, outputting two different representations from the first and second decoders, combining the two different representations, and outputting identified lanes, lane markings and the roadway.

## Claims

### Claim 1

A method using a multitask DNN architecture that includes an encoder, a first decoder and a second decoder, wherein the method comprises:
providing an image of a roadway by an optical detection device;
filtering the image by the encoder to produce an encoder output;
providing the encoder output to the first decoder and the second decoder;
in the first decoder, performing a semantic segmentation on the encoder output to produce a pixel classification of the image;
in the second decoder, performing a keypoint regression on the encoder output to produce keypoints of the image;
combining the pixel classification and the keypoints by fusion thereof to produce fused data;
processing the fused data to detect the roadway, lane markings on the roadway, and a lane of the roadway bounded by the lane markings; and
outputting roadway data identifying the roadway, the lane markings and the lanes.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising operating a driver assistance system of a vehicle in response to and dependent on the roadway data.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the encoder output supplies learned features as convolution filters.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the pixel classification produced by the first decoder supplies a pixel-precise classification of a drivable region within the image.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the drivable region is the roadway and/or the lane of the roadway.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the keypoints produced by the second decoder supply visible boundaries of the lane and/or of the roadway as continual pairs of values in image coordinates.

### Claim 7 (depends on 1)

The method according to claim 1, further comprising assessing an identification confidence for the lane and/or for the roadway based on the pixel classification produced by the first decoder and the keypoints produced by the second decoder.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the pixel classification produced by the first decoder defines a free area of the lane or of the roadway, and the keypoints produced by the second decoder define visible portions of boundaries of the lane or of the roadway.

### Claim 9 (depends on 8)

The method according to claim 8, further comprising interpolating concealed portions of the boundaries based on the keypoints.