# US 2019/0377986 A1 · Measuring crop residue from imagery using a machine-learned classification model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0377986 A1
- Document ID: US20190377986A1
- Status: Published application
- Kind: A1
- Application number: 16/002,321
- Filed: 2018-06-07
- Published: 2019-12-12
- Earliest claimed: 2018-06-07
- Assignees: CNH Industrial Canada, Ltd.; Autonomous Solutions, Inc.
- Applicants: CNH Industrial Canada, Ltd.; Autonomous Solutions, Inc.
- Inventors: Luca Ferrari; John H. Posselius; James W. Henry; Taylor C. Bybee
- Web page: https://patentgrep.com/patent/US20190377986A1

## CPC

- A01B 79/005: {Precision agriculture}
- G06F 18/241: relating to the classification model, e.g. parametric or non-parametric approaches
- G06F 18/2431: Multiple classes
- G06F 18/2433: Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
- G06V 20/188: {Vegetation}
- G06V 20/38: {Outdoor scenes}
- G06V 30/19173: {Classification techniques}
- A01B 19/10: Lifting or cleaning apparatus
- A01B 49/027: {with a rotating, soil working support element, e.g. a roller}
- A01B 63/32: operated by hydraulic or pneumatic means {without automatic control}
- A01B 73/044: {the axis being oriented in a longitudinal direction (A01B73/042 takes precedence)}
- G01B 11/285: {using photoelectric detection means}
- G06V 20/68: Food, e.g. fruit or vegetables
- A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS

## Abstract

The present disclosure provides systems and methods that measure crop residue in a field from imagery of the field. In particular, the present subject matter is directed to systems and methods that include or otherwise leverage a machine-learned crop residue classification model to determine a crop residue parameter value for a portion of a field based at least in part on imagery of such portion of the field captured by an imaging device. For example, the imaging device can be a camera positioned in a downward-facing direction and physically coupled to a work vehicle or an implement towed by the work vehicle through the field.

## Claims

### Claim 1

A computing system, comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store:
a machine-learned crop residue classification model configured to receive imagery and to process the imagery to output one or more crop residue classifications for the imagery; and
instructions that, when executed by the one or more processors, configure the computing system to perform operations, the operations comprising:
obtaining image data that depicts a portion of a field;
extracting a plurality of patches from the image data;
respectively inputting each of the plurality of patches from the image data into the machine-learned crop residue classification model;
receiving a respective crop residue classification for each of the plurality of patches as an output of the machine-learned crop residue classification model; and
determining a crop residue parameter value for the image data based at least in part on the respective crop residue classifications received for the plurality of patches.

### Claim 2 (depends on 1)

The computing system of claim 1, wherein:
the computing system is physically located on at least one of a work vehicle or an implement;
obtaining the image data of the field comprises obtaining image data captured by an imaging device that is physically located on at least one of the work vehicle or the implement; and
the operations further comprise controlling an operation of at least one of the work vehicle or the implement based at least in part on the crop residue parameter value determined for the image data.

### Claim 3 (depends on 1)

The computing system of claim 1, wherein the machine-learned crop residue classification model comprises an artificial deep neural network.

### Claim 4 (depends on 3)

The computing system of claim 3, wherein the artificial deep neural network comprises a convolutional neural network.

### Claim 5 (depends on 1)

The computing system of claim 1, wherein the machine-learned crop residue classification model comprises one or more of:
a support vector machine;
a Bayes classifier;
a K-nearest neighbor classifier; or
a texton-based classifier.

### Claim 6 (depends on 1)

The computing system of claim 1, wherein the machine-learned crop residue classification model comprises a binary classifier configured to output, for each patch, a binary crop residue classification that classifies such patch as residue or non-residue.

### Claim 7 (depends on 1)

The computing system of claim 1, wherein the machine-learned crop residue classification model comprises a multi-class classifier configured to output, for each patch, a multi-class crop residue classification that classifies such patch as non-residue or as one of a plurality of different types of residue.

### Claim 8 (depends on 1)

The computing system of claim 1, wherein:
the machine-learned crop residue classification model is configured to output a respective confidence score for the respective crop residue classification provided for each of the plurality of patches; and
determining the crop residue parameter value for the image data comprises determining the crop residue parameter value for the image data based at least in part on the respective confidence values associated with the respective crop residue classifications received for the plurality of patches.

### Claim 9 (depends on 1)

The computing system of claim 1, wherein the operations further comprise:
determining, for each of the plurality of patches, a respective weight based at least in part on an approximate ground area associated with such patch.

### Claim 10 (depends on 9)

The computing system of claim 9, wherein the operations further comprise:
determining the approximate ground area associated with each patch based at least in part on one or both of: imaging device calibration information or topography data associated with the field.

### Claim 11 (depends on 9)

The computing system of claim 9, wherein:
the machine-learned crop residue classification model is configured to output a respective confidence score for the respective crop residue classification provided for each of the plurality of patches; and
determining the crop residue parameter value for the image data comprises determining a weighted average of the confidence values weighted according to the respective weights.

### Claim 12 (depends on 1)

The computing system of claim 1, wherein extracting the plurality of patches from the image data comprises randomly selecting and extracting the plurality of patches from the image data.

### Claim 13 (depends on 1)

The computing system of claim 1, wherein the image data comprises a plurality of image frames.

### Claim 14 (depends on 1)

The computing system of claim 1, wherein the image data comprises a single image frame and wherein the operations further comprise:
obtaining a plurality of image frames that depict respective portions of the field;
performing the operations recited in claim 1 for each of the plurality of image frames to determine a plurality of frame-wise crop residue parameter values respectively for the plurality of image frames; and
determining a total crop residue parameter value for the field based at least in part on the plurality of frame-wise crop residue parameter values.

### Claim 15 (depends on 1)

The computing system of claim 1, wherein the plurality of patches comprise a subset of the image data.

### Claim 16

A computer-implemented method, comprising:
obtaining, by a computing system comprising one or more computing devices, image data that depicts a portion of a field;
extracting, by the computing system, a plurality of patches from the image data;
respectively, by the computing system, inputting each of the plurality of patches from the image data into a machine-learned crop residue classification model;
receiving, by the computing system, a respective crop residue classification for each of the plurality of patches as an output of the machine-learned crop residue classification model; and
determining, by the computing system, a crop residue parameter value for the image data based at least in part on the respective crop residue classifications received for the plurality of patches.

### Claim 17 (depends on 16)

The computer-implemented method of claim 16, wherein:
the computing system is physically located on at least one of a work vehicle or an implement;
obtaining, by the computing system, the image data of the field comprises obtaining, by the computing system, image data captured by an imaging device that is physically located on at least one of the work vehicle or the implement; and
the method further comprises controlling, by the computing system, an operation of at least one of the work vehicle or the implement based at least in part on the crop residue parameter value determined for the image data.

### Claim 18 (depends on 16)

The computer-implemented method of claim 16, wherein the machine-learned crop residue classification model comprises one or more of:
an artificial neural network;
a support vector machine;
a Bayes classifier;
a K-nearest neighbor classifier; or
a texton-based classifier.

### Claim 19 (depends on 16)

The computer-implemented method of claim 16, wherein:
the machine-learned crop residue classification model is configured to output a respective confidence score for the respective crop residue classification provided for each of the plurality of patches; and
determining, by the computing system, the crop residue parameter value for the image data comprises:
determining, by the computing system for each of the plurality of patches, a respective weight based at least in part on an approximate ground area associated with such patch; and
determining, by the computing system, a weighted average of the confidence values received for the plurality of patches, wherein the confidence values are weighted according to the respective weights.

### Claim 20

An agricultural work vehicle or implement, comprising:
one or more imaging devices; and
a controller comprising one or more processors and one or more non-transitory computer-readable media that collectively store:
a machine-learned crop residue classification model configured to receive imagery and to process the imagery to output one or more crop residue classifications for the imagery; and
instructions that, when executed by the one or more processors, configure the controller to perform operations, the operations comprising:
obtaining image data that depicts a portion of a field from the one or more imaging devices;
extracting a plurality of patches from the image data;
respectively inputting each of the plurality of patches from the image data into the machine-learned crop residue classification model;
receiving a respective crop residue classification for each of the plurality of patches as an output of the machine-learned crop residue classification model;
determining a crop residue parameter value for the image data based at least in part on the respective crop residue classifications received for the plurality of patches; and
controlling operation of one or more ground-engaging tools based at least in part on the crop residue parameter value determined for the image data.
