# US 2023/0274391 A1 · Crop yield prediction at field-level and pixel-level

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274391 A1
- Document ID: US20230274391A1
- Status: Published application
- Kind: A1
- Application number: 18/138,003
- Filed: 2023-04-21
- Published: 2023-08-31
- Earliest claimed: 2018-10-19
- Assignees: Mineral Earth Sciences LLC
- Applicants: Mineral Earth Sciences LLC
- Inventors: Cheng-en Guo; Wilson Zhao; Jie Yang; Zhiqiang Yuan
- Web page: https://patentgrep.com/patent/US20230274391A1

## CPC

- G06T 3/4007: based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403)
- A01D 41/127: Control or measuring arrangements specially adapted for combines
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06N 7/023: {Learning or tuning the parameters of a fuzzy system}
- G06N 20/00: Machine learning
- G06Q 10/04: Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem" (market predictions or forecasting for commercial activities G06Q30/0202)
- G06Q 50/02: Agriculture; Fishing; Forestry; Mining
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 7/00: Image analysis
- G06T 7/0016: {involving temporal comparison}
- G06T 7/143: involving probabilistic approaches, e.g. Markov random field [MRF] modelling
- G06V 10/82: using neural networks
- G06V 20/13: Satellite images
- G06V 20/188: {Vegetation}
- A01B 79/005: {Precision agriculture}
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/10032: Satellite or aerial image; Remote sensing
- G06T 2207/10048: Infrared image
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20221: Image fusion; Image merging
- G06T 2207/30181: Earth observation
- G06T 2207/30188: Vegetation; Agriculture
- G06T 2207/30192: Weather; Meteorology
- G06V 20/194: {using hyperspectral data, i.e. more or other wavelengths than RGB}
- Y02A 40/10: in agriculture
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A01D: HARVESTING; MOWING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21)
- Y02A: TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE

## Abstract

Implementations relate to crop yield prediction at the field- and pixel-level. In various implementations, a first temporal sequence of high-elevation digital images may be obtained that capture a first geographic area and are acquired over a first predetermined time interval while the first geographic area includes a particular crop. A first plurality of other data points may also be obtained that influence a ground truth crop yield of the first geographic area after the first predetermined time interval. The first plurality of other data points may be grouped into temporal chunks corresponding temporally with respective images of the first temporal sequence. The first temporal sequence and the temporal chunks of the first plurality of other data points may be applied, e.g., iteratively, as input across a machine learning model to estimate a crop yield of the first geographic area at the end of the first predetermined time interval.

## Claims

### Claim 1

A method implemented using one or more processors, comprising:
obtaining a first temporal sequence of high-elevation digital images,
wherein the first temporal sequence of high-elevation digital images capture a first geographic area at a first temporal frequency, and
wherein each high-elevation digital image of the first temporal sequence is captured at a first spatial resolution;
obtaining a second temporal sequence of high-elevation digital images,
wherein the second temporal sequence of high-elevation digital images capture the first geographic area at a second temporal frequency that is less than the first temporal frequency, and
wherein each high-elevation digital image of the second temporal sequence is captured at a second spatial resolution that is greater than the first spatial resolution;
selecting a given high-elevation digital image from the first temporal sequence that is captured during a time interval in which no high-elevation digital images of the second temporal sequence are available; and
fusing the given high-elevation digital image of the first temporal sequence with data ascertained from one or more high-elevation digital images of the second temporal sequence to generate a synthetic high-elevation digital image of the first geographic area at the second spatial resolution;
including the synthetic high-elevation digital image of the first geographic area as part of a third temporal sequence of high-elevation digital images that capture the geographic area over a predetermined time interval, wherein the first geographic area includes a first type of crop during the predetermine time interval;
iteratively applying the high-elevation digital images of the third temporal sequence as input across a machine learning model to generate first output indicative of an estimated crop yield of the first geographic area at the end of the predetermined time interval; and
causing one or more output components of one or more computing devices to audibly or visually convey the estimated crop yield of the first geographic area at the end of the predetermined time interval.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
obtaining a first plurality of other data points, wherein the first plurality of other data points influence a ground truth crop yield of the first geographic area after the first predetermined time interval, and wherein the first plurality of other data points are grouped into temporal chunks, each temporal chunk of the first plurality of other data points corresponding temporally with a respective high-elevation digital image of the third temporal sequence of high-elevation digital images;
wherein the temporal chunks of the first plurality of other data points are applied as input across the machine learning model to generate the first output.

### Claim 3 (depends on 2)

The method of claim 2, wherein the first plurality of other data points include one or more data points that are extrapolated from a geographic region that includes the first geographic area.

### Claim 4 (depends on 2)

The method of claim 2, wherein the first plurality of other data points include one or more data points that are extrapolated from another geographic area that is used to grow the first type of crop.

### Claim 5 (depends on 2)

The method of claim 2, wherein the first plurality of other data points include one or more data points that are inferred using expectation maximization.

### Claim 6 (depends on 2)

The method of claim 2, wherein the first plurality of other data points includes fertilizer applied to at least a portion of the first geographic area.

### Claim 7 (depends on 2)

The method of claim 2, wherein the first plurality of other data points includes pesticide applied to at least a portion of the first geographic area.

### Claim 8 (depends on 1)

The method of claim 1, wherein the fusing comprises, based on a first deviation of ground truth data forming the given high-elevation digital image of the first temporal sequence from data interpolated from the first temporal sequence of high-elevation digital images, predicting a second deviation from data interpolated from the second temporal sequence of high-elevation digital images.

### Claim 9 (depends on 8)

The method of claim 8, wherein the fusing further comprises, based on the given high-elevation digital image of the first temporal sequence selected from the first temporal sequence, and based on the second deviation, generating the synthetic high-elevation digital image of the geographic area at the second spectral resolution .

### Claim 10

A system comprising processor(s) and memory storing instructions that, in response to execution by the processor(s), cause the processor(s) to:
obtain a first temporal sequence of high-elevation digital images,
wherein the first temporal sequence of high-elevation digital images capture a first geographic area at a first temporal frequency, and
wherein each high-elevation digital image of the first temporal sequence is captured at a first spatial resolution;
obtain a second temporal sequence of high-elevation digital images,
wherein the second temporal sequence of high-elevation digital images capture the first geographic area at a second temporal frequency that is less than the first temporal frequency, and
wherein each high-elevation digital image of the second temporal sequence is captured at a second spatial resolution that is greater than the first spatial resolution;
select a given high-elevation digital image from the first temporal sequence that is captured during a time interval in which no high-elevation digital images of the second temporal sequence are available; and
fuse the given high-elevation digital image of the first temporal sequence with data ascertained from one or more high-elevation digital images of the second temporal sequence to generate a synthetic high-elevation digital image of the first geographic area at the second spatial resolution;
include the synthetic high-elevation digital image of the first geographic area as part of a third temporal sequence of high-elevation digital images that capture the geographic area over a predetermined time interval, wherein the first geographic area includes a first type of crop during the predetermine time interval;
iteratively apply the high-elevation digital images of the third temporal sequence as input across a machine learning model to generate first output indicative of an estimated crop yield of the first geographic area at the end of the predetermined time interval; and
cause one or more output components of one or more computing devices to audibly or visually convey the estimated crop yield of the first geographic area at the end of the predetermined time interval.

### Claim 11 (depends on 10)

The system of claim 10, further comprising instructions to:
obtain a first plurality of other data points, wherein the first plurality of other data points influence a ground truth crop yield of the first geographic area after the first predetermined time interval, and wherein the first plurality of other data points are grouped into temporal chunks, each temporal chunk of the first plurality of other data points corresponding temporally with a respective high-elevation digital image of the third temporal sequence of high-elevation digital images;
wherein the temporal chunks of the first plurality of other data points are applied as input across the machine learning model to generate the first output.

### Claim 12 (depends on 11)

The system of claim 11, wherein the first plurality of other data points include one or more data points that are extrapolated from a geographic region that includes the first geographic area.

### Claim 13 (depends on 11)

The system of claim 11, wherein the first plurality of other data points include one or more data points that are extrapolated from another geographic area that is used to grow the first type of crop.

### Claim 14 (depends on 11)

The system of claim 11, wherein the first plurality of other data points include one or more data points that are inferred using expectation maximization.

### Claim 15 (depends on 11)

The system of claim 11, wherein the first plurality of other data points includes fertilizer applied to at least a portion of the first geographic area.

### Claim 16 (depends on 11)

The system of claim 11, wherein the first plurality of other data points includes pesticide applied to at least a portion of the first geographic area.

### Claim 17 (depends on 10)

The system of claim 10, wherein the instructions to fuse include instructions to, based on a first deviation of ground truth data forming the given high-elevation digital image of the first temporal sequence from data interpolated from the first temporal sequence of high-elevation digital images, predict a second deviation from data interpolated from the second temporal sequence of high-elevation digital images.

### Claim 18 (depends on 17)

The system of claim 17, wherein the synthetic high-elevation digital image of the geographic area at the second spectral resolution is generated based on the given high-elevation digital image of the first temporal sequence selected from the first temporal sequence, and based on the second deviation.

### Claim 19

At least one non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to:
obtain a first temporal sequence of high-elevation digital images,
wherein the first temporal sequence of high-elevation digital images capture a first geographic area at a first temporal frequency, and
wherein each high-elevation digital image of the first temporal sequence is captured at a first spatial resolution;
obtain a second temporal sequence of high-elevation digital images,
wherein the second temporal sequence of high-elevation digital images capture the first geographic area at a second temporal frequency that is less than the first temporal frequency, and
wherein each high-elevation digital image of the second temporal sequence is captured at a second spatial resolution that is greater than the first spatial resolution;
select a given high-elevation digital image from the first temporal sequence that is captured during a time interval in which no high-elevation digital images of the second temporal sequence are available; and
fuse the given high-elevation digital image of the first temporal sequence with data ascertained from one or more high-elevation digital images of the second temporal sequence to generate a synthetic high-elevation digital image of the first geographic area at the second spatial resolution;
include the synthetic high-elevation digital image of the first geographic area as part of a third temporal sequence of high-elevation digital images that capture the geographic area over a predetermined time interval, wherein the first geographic area includes a first type of crop during the predetermine time interval;
iteratively apply the high-elevation digital images of the third temporal sequence as input across a machine learning model to generate first output indicative of an estimated crop yield of the first geographic area at the end of the predetermined time interval; and
cause one or more output components of one or more computing devices to audibly or visually convey the estimated crop yield of the first geographic area at the end of the predetermined time interval.

### Claim 20 (depends on 19)

The at least one non-transitory computer-readable medium of claim 19, further comprising instructions to:
obtain a first plurality of other data points, wherein the first plurality of other data points influence a ground truth crop yield of the first geographic area after the first predetermined time interval, and wherein the first plurality of other data points are grouped into temporal chunks, each temporal chunk of the first plurality of other data points corresponding temporally with a respective high-elevation digital image of the third temporal sequence of high-elevation digital images;
wherein the temporal chunks of the first plurality of other data points are applied as input across the machine learning model to generate the first output.
