# US 8,150,574 B2 · Method and system for guiding a vehicle with vision-based adjustment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,150,574 B2
- Document ID: US8150574B2
- Status: Granted
- Kind: B2
- Application number: 11/755,992
- Filed: 2007-05-31
- Published: 2008-03-13
- Granted: 2012-04-03
- Earliest claimed: 2005-01-04
- Assignees: Deere & Company
- Inventors: Shufeng Han; Terence Daniel Pickett; John Franklin Reid
- Web page: https://patentgrep.com/patent/US8150574B2

## CPC

- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- A01B 69/007: {Steering or guiding of agricultural vehicles, e.g. steering of the tractor to keep the plough in the furrow}
- G01S 19/485: {whereby the further system is an optical system or imaging system}
- G05D 1/0278: {using satellite positioning signals, e.g. GPS}
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Preliminary guidance data is determined for the vehicle during an evaluation time window. A vision module collects vision data from a vision module during the evaluation time window. Vision guidance data is determined from the collected vision data. A vision quality estimator estimates vision quality data for at least one of the vision data and the vision guidance data during the evaluation time window. The vision quality data is based on a regression path and density grid points. An adjuster adjusts the preliminary guidance data to a revised guidance data based on the vision guidance data such that the revised guidance data is registered with or generally coextensive with the vision guidance data, if the vision quality data exceeds a minimum threshold.

## Claims

### Claim 1

A method for guiding a vehicle, the method comprising:
determining preliminary guidance data for the vehicle based a location-determining receiver associated with the vehicle during an evaluation time window;
collecting vision data from a vision module associated with the vehicle during the evaluation time window;
determining vision guidance data from the collected vision data;
estimating vision quality data for the vision guidance data during the evaluation time window, based on organizing a collected image into crop pixels and non-crop pixels, defining a density grid based on the number of crop pixels within each grid cell, defining a density grid points as those grid cells whose values are greater than a threshold, arranging a regression path based on the density grid points, the regression path representing a possible position and possible orientation of a crop row or a windrow, determining a confidence index as vision quality data based on a product of a moment index and a cluster index for application to a vehicle controller or an adjuster for steering a vehicle;
adjusting the preliminary guidance data to a revised guidance data based on the vision guidance data such that the revised guidance data is registered with or generally coextensive with the vision guidance data, if the vision quality data exceeds a minimum threshold; and guiding the vehicle in accordance with the revised guidance data.

### Claim 2 (depends on 1)

The method according to claim 1 wherein the preliminary guidance data and revised guidance data each comprise at least one of heading data, off-track data, and curvature data.

### Claim 3 (depends on 1)

The method according to claim 1 wherein the collecting vision data comprises collecting the vision data via a stereo vision imaging system.

### Claim 4 (depends on 1)

The method according to claim 1 wherein the confidence index is a product of a moment index, CM, and a cluster index, CC as determined in accordance with the following equations:
where CM is the moment index given by
C
M
=
1
-
M
M
max
,
M
is moment of density grid points around a regression line, and Mmax is maximum moments;
Cc is the cluster index given by
C
C
=
0.8
+
0.2
σ
d
,
where
⁢
⁢
σ
d
=
∑
i
=
1
n
⁢
(
d
i
-
d
_
)
2
n
-
1
.
where di is an orthogonal distance from a regression path to the density grid point;
d is mean distance of the density grid points to the regression path;
n is a total number of density grid points; and
σd represents a variance of the density grid points.

### Claim 5

A method for estimating a vision quality data for guidance of a vehicle, the method comprising:
organizing a collected image into crop pixels and non-crop pixels;
defining a density grid based on the number of crop pixels within each grid cell;
defining a density grid points as those grid cells whose values are greater than a threshold;
arranging a regression path based on the density grid points, the regression path representing a possible position and possible orientation of a crop row or a windrow;
determining vision quality data based on evaluating how the defined density grid points are distributed about the regression path; and determining a cluster index as an inverse function of a standard deviation of the density grid points, for application to a vehicle controller or an adjuster for steering the vehicle.

### Claim 6 (depends on 5)

The method according to claim 5 wherein the confidence index is a product of a moment index, CM, and a cluster index, CC, as determined in accordance with the following equation:
where CM is the moment index given by
C
M
=
1
-
M
M
max
,
M
is moment of density grid points around a regression line, and Mmax is maximum moment;
Cc is the cluster index given by
C
c
=
0.8
+
0.2
σ
d
,
where
σ
d
=
∑
i
=
1
n
⁢
⁢
(
d
i
-
d
_
)
2
n
-
1
.
where di is an orthogonal distance from a regression path to the density grid point;
d is mean distance of the density grid points to the regression path;
n is a total number of density grid points; and
σd presents a variance of the density grid points.

### Claim 7 (depends on 5)

The method according to claim 5 further comprising:
collecting the image data via a stereo vision imaging system.

### Claim 8

A system for estimating a vision quality data for guidance of a vehicle, the system comprising:
an organizer for organizing a collected image into crop pixels and non-crop pixels;
a density grid definer for defining a density grid based on the number of crop pixels within each grid cell;
a filter for defining a density grid points as those grid cells whose values are greater than a threshold;
an arranger for arranging a regression path based on the density grid points, the regression path representing a possible position and possible orientation of a crop row or a windrow; and
an estimator arranged for determining vision quality data based on evaluating how the defined density grid points are distributed about the regression path, and to determine a cluster index as an inverse function of a standard deviation of the density grid points, for application to a vehicle controller or an adjuster for steering the vehicle.

### Claim 9 (depends on 8)

The system according to claim 8 wherein the confidence index is a product of a moment index, CM, and a cluster index, CC, as determined in accordance with the following equation:
where CM is the moment index given by
C
M
=
1
-
M
M
max
,
M
is moment of density grid points around a regression line, and Mmax is maximum moment;
CC is the cluster index given by
C
c
=
0.8
+
0.2
σ
d
,
where
σ
d
=
∑
i
=
1
n
⁢
⁢
(
d
i
-
d
_
)
2
n
-
1
where di is an orthogonal distance from a regression path to the density point;
d is mean distance of the density grid points to the regression path:
n is a total number of density grid points; and
σd presents a variance of the density grid points.

### Claim 10 (depends on 8)

The system according to claim 8 further comprising:
a stereo vision imaging system for collecting the image data.

### Claim 11

A system for estimating a vision quality data for guidance of a vehicle, the system comprising:
an organizer for organizing a collected image into crop pixels and non-crop pixels;
a density grid definer for defining a density grid based on the number of crop pixels within each grid cell;
a filter for defining a density grid points as those grid cells whose values are greater than a threshold;
an arranger for arranging a regression path based on the density grid points, the regression path representing a possible position and possible orientation of a crop row or a windrow; and
an estimator for determining a confidence index as vision quality data based on the defined density grid points and the regression path, the confidence index comprising a product of the moment index and the cluster index for application to a vehicle controller or an adjuster for steering a vehicle.