# US 2024/0202966 A1 · Crop row detection system, agricultural machine having a crop row detection system, and method of crop row detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202966 A1
- Document ID: US20240202966A1
- Status: Published application
- Kind: A1
- Application number: 18/392,251
- Filed: 2023-12-21
- Published: 2024-06-20
- Earliest claimed: 2021-06-29
- Assignees: Kubota Corporation
- Applicants: Kubota Corporation
- Inventors: Shunsuke MIYASHITA; Toru TAMBO; Mitsuaki NAGAO
- Web page: https://patentgrep.com/patent/US20240202966A1

## CPC

- G06T 7/73: using feature-based methods
- A01B 69/001: {Steering by means of optical assistance, e.g. television cameras (steering devices for road marking vehicles E01C23/163)}
- A01B 69/008: {automatic}
- B62D 15/025: {Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation (B62D1/28 takes precedence)}
- G05D 1/2435: {Extracting 3D information}
- G05D 1/646: Following a predefined trajectory, e.g. a line marked on the floor or a flight path
- G06T 7/90: Determination of colour characteristics
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/56: relating to colour
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G05D 2101/20: using external object recognition
- G05D 2105/15: for harvesting, sowing or mowing in agriculture or forestry
- G05D 2107/21: {Farming, e.g. fields, pastures or barns}
- G05D 2109/10: Land vehicles
- G05D 2111/10: Optical signals
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/20072: Graph-based image processing
- G06T 2207/30188: Vegetation; Agriculture
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21)
- B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A crop row detection system includes a camera mounted to an agricultural machine to image a ground surface traveled by the agricultural machine to acquire time-series color images including at least a portion of the ground surface, and a processor configured or programmed to (i) perform image processing for the time-series color images, (ii) generate, from the time-series color images, an enhanced image in which a color of a crop row for detection is enhanced to provide an enhanced image, (iii) generate from the enhanced image a plan view image as viewed from above the ground surface, the plan view image being classified into first pixels having a color index value for the crop row equal to or greater than a threshold and second pixels having the color index value below the threshold, and (iv) determine positions of edge lines of the crop row based on the color index values of the first pixels.

## Claims

### Claim 1

A crop row detection system comprising:
a camera mounted to an agricultural machine to image a ground surface traveled by the agricultural machine to acquire time-series color images including at least a portion of the ground surface; and
a processor configured or programmed to:
perform image processing for the time-series color images;
generate from the time-series color images an enhanced image in which a color of a crop row for detection is enhanced to provide an enhanced image;
generate from the enhanced image a plan view image as viewed from above the ground surface, the plan view image being classified into first pixels having a color index value for the crop row equal to or greater than a threshold and second pixels having the color index value below the threshold; and
determine positions of edge lines of the crop row based on the color index values of the first pixels.

### Claim 2 (depends on 1)

The crop row detection system of claim 1, wherein the processor is configured or programmed to:
total the color index values of the first pixels along a plurality of scanning lines in the plan view image to determine total values, and generate a histogram in which positions of the scanning lines and the total values are associated; and
determine the positions of the edge lines of the crop row based on the histogram.

### Claim 3 (depends on 2)

The crop row detection system of claim 2, wherein the processor is configured or programmed to refer to the histogram, and determine the positions of the edge lines of the crop row from predetermined positions on opposite sides of a peak of the total values.

### Claim 4 (depends on 1)

The crop row detection system of claim 1, wherein the processor is configured or programmed to split an entirety or a portion of the plan view image into a plurality of blocks, and determine the positions of the edge lines for each of the plurality of blocks.

### Claim 5 (depends on 4)

The crop row detection system of claim 4, wherein
in the plan view image, the plurality of blocks have belt shapes that are continuous along a horizontal direction in the image or a vertical direction in the image; and
the processor is configured or programmed to determine the edge lines of the crop row based on a belt shape in a direction that is different from a traveling direction of the agricultural machine.

### Claim 6 (depends on 4)

The crop row detection system of claim 4, wherein, based on the positions of the edge lines in each of the plurality of blocks, the processor is configured or programmed to determine a direction in which the crop row extends.

### Claim 7 (depends on 1)

The crop row detection system of claim 1, wherein
the plan view image is an overhead view image in which a reference plane extending along the ground surface is viewed directly from above along a normal direction of the reference plane; and
the processor is configured or programmed to generate the overhead view image through homography transformation from the time-series color images or preprocessed images of the time-series color images.

### Claim 8 (depends on 7)

The crop row detection system of claim 7, wherein the reference plane is offset upward by a predetermined distance that is set in accordance with bumps and dents on the ground surface having the crops being planted thereon, from bottoms of the bumps and dents of the ground surface.

### Claim 9 (depends on 1)

The crop row detection system of claim 1, wherein the processor is configured or programmed to generate and output a target path based on the positions of the edge lines of the crop row.

### Claim 10 (depends on 1)

An agricultural machine comprising:
the crop row detection system of claim 1;
a wheel; and
an automatic steering controller configured or programmed to control a steering angle of the wheel based on the positions of the edge lines of the crop row as determined by the crop row detection system.

### Claim 11 (depends on 10)

The agricultural machine of claim 10, wherein, based on the time-series color images, the processor is configured or programmed to monitor a positional relationship between the edge lines of the crop row and the wheel, and supply a positional error signal to the automatic steering controller.

### Claim 12

A computer-implemented method of crop row detection, the method of crop row detection causing a computer to execute:
acquiring from a camera mounted to an agricultural machine, time-series color images by imaging a ground surface that is traveled by the agricultural machine, the time-series color images including at least a portion of the ground surface;
generating from the time-series color images an enhanced image in which a color of a crop row for detection is enhanced;
generating from the enhanced image a plan view image as viewed from above the ground surface, the plan view image being classified into first pixels having a color index value for the crop row equal to or greater than a threshold and second pixels of having the color index value below the threshold; and
determining positions of edge lines of the crop row based on the color index values of the first pixels.
