# US 2024/0389494 A1 · Automated steering by machine vision

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0389494 A1
- Document ID: US20240389494A1
- Status: Published application
- Kind: A1
- Application number: 18/691,625
- Filed: 2022-08-30
- Published: 2024-11-28
- Earliest claimed: 2022-08-30
- Assignees: Topcon Positioning Systems, Inc.
- Applicants: Topcon Positioning Systems, Inc.
- Inventors: Mikhail Yurievich VOROBIEV; Alexey Vladimirovich KALMYKOV; Nikita Andreevich KUTKIN
- Web page: https://patentgrep.com/patent/US20240389494A1

## CPC

- A01B 69/008: {automatic}
- A01B 69/001: {Steering by means of optical assistance, e.g. television cameras (steering devices for road marking vehicles E01C23/163)}
- G01C 21/20: Instruments for performing navigational calculations (G01C21/24, G01C21/26 take precedence)
- G05D 1/2435: {Extracting 3D information}
- G05D 1/646: Following a predefined trajectory, e.g. a line marked on the floor or a flight path
- G06V 20/188: {Vegetation}
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G05D 2105/15: for harvesting, sowing or mowing in agriculture or forestry
- G05D 2107/21: {Farming, e.g. fields, pastures or barns}
- G05D 2109/10: Land vehicles
- G05D 2111/10: Optical signals
- G05D 2111/64: {taken simultaneously from spaced apart sensors, e.g. stereovision}
- A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21)
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for automated steering by machine vision receives a point cloud that is generated using a stereo camera. A location of a row is determined based on the point cloud and a steering angle is generated based on the location of the row. The center of the row is detected using a Hough transform detection algorithm and a horizontal projection of the point cloud.

## Claims

### Claim 1

A method for automatic steering of an agricultural machine comprising:
receiving point cloud data from a camera;
determining a location of a row of plants based on the point cloud data; and
generating a steering angle for the agricultural machine based on the location of the row.

### Claim 2 (depends on 1)

The method of claim 1, wherein the determining comprises:
determining a centerline of the row based on a Hough transform detection algorithm.

### Claim 3 (depends on 2)

The method of claim 2, wherein the Hough transform detection algorithm determines the centerline of the row based on a horizontal projection of the point cloud data.

### Claim 4 (depends on 3)

The method of claim 3, wherein the horizontal projection of the point cloud data is based on points of the point cloud that are higher than a ground plane found using a random sample consensus algorithm.

### Claim 5 (depends on 4)

The method of claim 4, wherein the determining the centerline of the row is further based on a front projection of the point cloud data.

### Claim 6 (depends on 1)

The method of claim 1, wherein the generating a steering angle is further based on a heading error.

### Claim 7 (depends on 6)

The method of claim 6, wherein the heading error is an angle between a longitudinal axis of the agricultural machine and a median line.

### Claim 8 (depends on 7)

The method of claim 7, wherein the generating a steering angle is further based on an Xtrack.

### Claim 9 (depends on 8)

The method of claim 8, wherein the Xtrack is a distance from the median line to a center of a rear wheel axis of the agricultural machine.

### Claim 10 (depends on 9)

The method of claim 9, wherein the median line is a centerline of the row.

### Claim 11 (depends on 9)

The method of claim 9, wherein the median line is a centerline located between the row and an adjacent row parallel to, and offset from, the row.

### Claim 12

An apparatus comprising:
a steering actuator;
a camera; and
a steering controller in communication with the steering actuator and the camera, the steering controller configured to perform operations comprising:
receiving point cloud data from the camera;
determining a location of a row of plants based on the point cloud data; and
generating a steering angle for an agricultural machine based on the location of the row.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the determining comprises:
determining a centerline of the row based on a Hough transform detection algorithm.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the Hough transform detection algorithm determines the centerline of the row based on a horizontal projection of the point cloud data.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the horizontal projection of the point cloud data is based on points of the point cloud that are higher than a ground plane found using a random sample consensus algorithm.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the determining the centerline of the row is further based on a front projection of the point cloud data.

### Claim 17 (depends on 12)

The apparatus of claim 12, wherein the generating a steering angle is further based on a heading error.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the generating a steering angle is further based on an Xtrack.

### Claim 19

A computer readable medium storing computer program instructions for automatic steering of an agricultural machine, which, when executed on a processor, cause the processor to perform operations comprising:
receiving point cloud data from a camera;
determining a location of a row of plants based on the point cloud data; and
generating a steering angle for the agricultural machine based on the location of the row.

### Claim 20 (depends on 19)

The computer readable medium of claim 19, wherein the determining comprises:
determining a centerline of the row based on a Hough transform detection algorithm.

### Claim 21 (depends on 20)

The computer readable medium of claim 20, wherein the Hough transform detection algorithm determines the centerline of the row based on a horizontal projection of the point cloud data.

### Claim 22 (depends on 21)

The computer readable medium of claim 21, wherein the horizontal projection of the point cloud data is based on points of the point cloud that are higher than a ground plane found using a random sample consensus algorithm.

### Claim 23 (depends on 22)

The computer readable medium of claim 22, wherein the determining the centerline of the row is further based on a front projection of the point cloud data.

### Claim 24 (depends on 19)

The computer readable medium of claim 19, wherein the generating a steering angle is further based on a heading error.

### Claim 25 (depends on 24)

The computer readable medium of claim 24, wherein the generating a steering angle is further based on an Xtrack.
