# US 9,751,529 B2 · Lane sensing through lane marker identification for lane centering/keeping

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,751,529 B2
- Document ID: US9751529B2
- Status: Granted
- Kind: B2
- Application number: 13/156,974
- Filed: 2011-06-09
- Published: 2012-12-13
- Granted: 2017-09-05
- Earliest claimed: 2011-06-09
- Assignees: GM Global Technology Operations LLC
- Applicants: Wende Zhang; Bakhtiar Brian Litkouhi; Jin-Woo Lee
- Inventors: Wende Zhang; Bakhtiar Brian Litkouhi; Jin-Woo Lee
- Web page: https://patentgrep.com/patent/US9751529B2

## CPC

- B60W 30/12: Lane keeping
- B60W 10/20: including control of steering systems
- B60W 40/06: Road conditions
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2710/202: Steering torque
- B60W 2710/207: Steering angle of wheels
- B60W 2720/14: Yaw
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

A method for adjusting a vehicle's position in a roadway lane. A camera mounted on the vehicle generates a current image of the lane and the method identifies a current lane-center line in the current image. A reference image is generated and the method identifies a reference lane-center line in the reference image. The method then calculates an error between the current lane-center line and the reference lane-center line and provide steering commands to adjust the position of the vehicle so that the error is reduced.

## Claims

### Claim 1

A method for correcting a position of a vehicle in a roadway lane, said method comprising:
generating a reference image of the roadway lane using a camera mounted on the vehicle, where the reference image is a two-dimensional image;
identifying a reference lane-center line on the two-dimensional reference image;
generating a current image of the lane using the camera, where the current image is a two-dimensional image;
identifying a current lane-center line on the two-dimensional current image;
calculating an error between the current lane-center line and the reference lane-center line; and
adjusting the position of the vehicle with a controller so that the error is reduced, wherein identifying the current lane center line and the reference lane-center line includes identifying the current lane-center line and the reference lane-center line by lane-center parameters including an angle of the lane-center line and a location where the lane-center line touches an edge line of the image.

### Claim 2 (depends on 1)

The method of claim 1 wherein adjusting the position of the vehicle includes providing a steering adjustment command signal to a vehicle steering system.

### Claim 3 (depends on 1)

The method of claim 1 wherein identifying the current lane-center line includes using camera-based lane marker detection to identify a left lane-boundary and a right lane-boundary in the current image, and then averaging the left and right lane-boundaries to identify the lane-center line.

### Claim 4 (depends on 1)

The method of claim 1 wherein generating the reference image includes generating the reference image when the vehicle is in a geometric center of the lane or the geometric center of the lane plus a bias.

### Claim 5 (depends on 1)

The method of claim 1 wherein generating the reference image includes identifying the reference image by machine learning techniques that are based on a driver's preferred bias.

### Claim 6 (depends on 1)

The method of claim 1 further comprising determining a desired driver offset relative to a center line in the roadway lane, and modifying the reference lane-center line based on the offset.

### Claim 7 (depends on 1)

The method of claim 1 wherein adjusting the position of the vehicle includes generating a sequence of steering angle commands that minimizes a cost function J as:
J
=
∫
0
Δ
⁢
⁢
T
⁢
{
[
y
L
φ
]
err
·
Q
⁡
(
x
)
·
[
y
L
φ
]
err
+
δ
·
R
⁡
(
t
)
·
δ
}
⁢
d
⁢
⁢
t
where yL,err is an offset error (yL,ref−yL,current), φerr is a heading angle error (φref−φcurrent), yL,ref and φref are the reference lane-center parameters, and yL,current and φcurrent are the current lane-center parameters, Q(t) and R(t) are weighting factors, and δ is a steering command angle.

### Claim 8 (depends on 7)

The method of claim 7 wherein adjusting the position of the vehicle includes generating a sequence of steering torques or yaw moments instead of steering angle commands to minimize the cost function J.

### Claim 9

A lane centering system for centering a vehicle in a roadway lane that the vehicle is traveling in, said system comprising:
a camera that generates images of the roadway lane, where the images are two-dimensional images;
a reference image processor that identifies a reference lane-center line on a two-dimensional reference image selected from the camera that is identified as an image when the vehicle was in a geometric center of the lane plus a predetermined bias using machine learning techniques:
a current image processor that identifies a current lane-center line on a two-dimensional current image from the camera;
a comparator that compares the reference lane-center line and the current lane-center line and provides an error therebetween; and
a lane-centering controller that receives the error and calculates a steering command that centers the vehicle in the lane by reducing the error.

### Claim 10 (depends on 9)

The centering system of claim 9 further comprising a vehicle steering system, said vehicle steering system receiving the steering command from the lane-centering controller to steer the vehicle in the lane.

### Claim 11 (depends on 9)

The centering system of claim 9 wherein the reference image processor and the current image processor identify the current lane-center line and the reference lane-center line by lane-center parameters that include an angle of the lane-center line and a location that the lane-center line touches an edge line of the image.

### Claim 12 (depends on 9)

The centering system of claim 9 wherein the reference image processor and the current image processor use camera-based lane marker detection to identify a left lane-boundary and a right lane-boundary in the reference image and the current image, and then average the left and right lane-boundaries to identify the lane-center line.

### Claim 13 (depends on 9)

The centering system of claim 9 wherein the reference image processor selects the reference image from the camera when the vehicle is in a geometric center of the lane or the geometric center of the lane plus a bias.

### Claim 14 (depends on 9)

The centering system of claim 9 wherein the reference image processor selects the reference image from the camera by machine learning techniques when the vehicle is in a geometric center of the lane.

### Claim 15 (depends on 9)

The centering system of claim 9 wherein the reference image processor selects the reference image from the camera by machine learning techniques that are for the same driver representing that driver's preferred bias.

### Claim 16 (depends on 9)

The centering system of claim 9 further comprising a preference processor that determines a desired driver offset relative to a center of the lane, said preference processor providing the driver offset to the reference image processor.

### Claim 17 (depends on 16)

The centering system of claim 16 wherein the preference processor generates the reference image when the vehicle is in a preferred reference location within the lane.

### Claim 18

A method for correcting a position of a vehicle in a roadway lane, said method comprising:
generating a reference image of the roadway lane using a camera mounted on the vehicle, where the reference image is a two-dimensional image;
identifying a reference lane-center line on the two-dimensional reference image;
generating a current image of the lane using the camera, where the current image is a two-dimensional image;
identifying a current lane-center line on the two-dimensional current image;
calculating an error between the current lane-center line and the reference lane-center line; and
adjusting the position of the vehicle with a controller so that the error is reduced, wherein generating the reference image includes identifying the reference image by machine learning techniques when the vehicle is in a geometric center of the lane.