# US 2023/0274541 A1 · Aggregate trait estimation for agricultural plots

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274541 A1
- Document ID: US20230274541A1
- Status: Published application
- Kind: A1
- Application number: 17/681,062
- Filed: 2022-02-25
- Published: 2023-08-31
- Earliest claimed: 2022-02-25
- Assignees: X Development LLC
- Applicants: X Development LLC
- Inventors: Zachary Beaver; Ming Zheng; Chuan Du; Tiffany Cai
- Web page: https://patentgrep.com/patent/US20230274541A1

## CPC

- G06V 20/188: {Vegetation}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- A01B 79/005: {Precision agriculture}
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/803: {of input or preprocessed data}
- G06V 10/811: {the classifiers operating on different input data, e.g. multi-modal recognition}
- G06V 10/82: using neural networks
- G06V 10/95: {structured as a network, e.g. client-server architectures}
- G06V 20/68: Food, e.g. fruit or vegetables
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21)

## Abstract

Implementations are described herein for estimating aggregate trait values for agricultural plots. In various implementations, a plurality of images depicting crops in an agricultural plot may be processed using a feature extraction portion of aggregate trait estimation model to generate a corresponding plurality of embeddings. The plurality of embeddings may be combined to obtain a unified representation of the crops in the agricultural plot. Using a prediction portion of the aggregate trait estimation model, the unified representation of the crops may be processed to estimate, as a direct output of the aggregate trait estimation model, an aggregate trait value of the crops in the agricultural plot. Data indicative of the estimated aggregate trait value of the crops in the agricultural plot may be output at a computing device.

## Claims

### Claim 1

A method implemented using one or more processors and comprising:
processing, using a feature extraction portion of an aggregate trait estimation model, a plurality of images depicting crops in an agricultural plot to generate a corresponding plurality of embeddings;
combining the plurality of embeddings to obtain a unified representation of the crops in the agricultural plot;
processing, using a prediction portion of the aggregate trait estimation model, the unified representation of the crops to estimate, as a direct output of the aggregate trait estimation model, an aggregate trait value of the crops in the agricultural plot; and
causing data indicative of the estimated aggregate trait value of the crops in the agricultural plot to be output at a computing device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising capturing the plurality of images as a first plurality of images using a first vision sensor carried across the agricultural plot by an agricultural vehicle.

### Claim 3 (depends on 2)

The method of claim 2, further comprising capturing a second plurality of images using a second vision sensor carried across the agricultural plot by the agricultural vehicle, wherein the first vision sensor captures the crops from a first angle and the second vision sensor captures the crops from a second angle that is different than the first angle.

### Claim 4 (depends on 3)

The method of claim 3, wherein the first angle comprises a side view of the crops and the second angle comprises a top-down view of the crops.

### Claim 5 (depends on 3)

The method of claim 3, wherein the aggregate trait value comprises a first aggregate trait value, the aggregate trait estimation model comprises a first aggregate trait estimation model, and the method further comprises:
processing, using a feature extraction portion of a second aggregate trait estimation model, the second plurality of images to generate a corresponding second plurality of embeddings;
combining the second plurality of embeddings to obtain a second unified representation of the crops in the agricultural plot; and
processing, using a prediction portion of the second aggregate trait estimation model, the second unified representation of the crops to estimate, as a direct output of the second aggregate trait estimation model, a second aggregate trait value of the crops in the agricultural plot.

### Claim 6 (depends on 5)

The method of claim 5, further comprising calculating a final aggregate trait value of the crops in the agricultural plot based on the first and second aggregate trait values.

### Claim 7 (depends on 1)

The method of claim 1, wherein the combining comprises concatenating or averaging the plurality of embeddings.

### Claim 8 (depends on 1)

The method of claim 1, wherein the estimated aggregate trait value comprises a probability within a normalized range that corresponds to non-normalized, ground truth aggregate trait values observed in a plurality of fields.

### Claim 9

A system comprising one or more processors and memory storing instructions that, in response to execution of the instructions, cause the one or more processors to:
process, using a feature extraction portion of an aggregate trait estimation model, a plurality of images depicting crops in an agricultural plot to generate a corresponding plurality of embeddings;
combine the plurality of embeddings to obtain a unified representation of the crops in the agricultural plot;
process, using a prediction portion of the aggregate trait estimation model, the unified representation of the crops to estimate, as a direct output of the aggregate trait estimation model, an aggregate trait value of the crops in the agricultural plot; and
cause data indicative of the estimated aggregate trait value of the crops in the agricultural plot to be output at a computing device.

### Claim 10 (depends on 9)

The system of claim 9, further comprising instructions to capture the plurality of images as a first plurality of images using a first vision sensor carried across the agricultural plot by an agricultural vehicle.

### Claim 11 (depends on 10)

The system of claim 10, further comprising instruction to capture a second plurality of images using a second vision sensor carried across the agricultural plot by the agricultural vehicle, wherein the first vision sensor captures the crops from a first angle and the second vision sensor captures the crops from a second angle that is different than the first angle.

### Claim 12 (depends on 11)

The system of claim 11, wherein the first angle comprises a side view of the crops and the second angle comprises a top-down view of the crops.

### Claim 13 (depends on 11)

The system of claim 11, wherein the aggregate trait value comprises a first aggregate trait value, the aggregate trait estimation model comprises a first aggregate trait estimation model, and the memory comprises instructions to:
process, using a feature extraction portion of a second aggregate trait estimation model, the second plurality of images to generate a corresponding second plurality of embeddings;
combine the second plurality of embeddings to obtain a second unified representation of the crops in the agricultural plot; and
process, using a prediction portion of the second aggregate trait estimation model, the second unified representation of the crops to estimate, as a direct output of the second aggregate trait estimation model, a second aggregate trait value of the crops in the agricultural plot.

### Claim 14 (depends on 13)

The system of claim 13, further comprising instructions to calculate a final aggregate trait value of the crops in the agricultural plot based on the first and second aggregate trait values.

### Claim 15 (depends on 9)

The system of claim 9, wherein the plurality of embeddings are combined by concatenation or averaging.

### Claim 16 (depends on 9)

The system of claim 9, wherein the estimated aggregate trait value comprises a probability within a normalized range that corresponds to non-normalized, ground truth aggregate trait values observed in a plurality of fields.

### Claim 17

A non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by a processor, cause the processor to:
process, using a feature extraction portion of an aggregate trait estimation model, a plurality of images depicting crops in an agricultural plot to generate a corresponding plurality of embeddings;
combine the plurality of embeddings to obtain a unified representation of the crops in the agricultural plot;
process, using a prediction portion of the aggregate trait estimation model, the unified representation of the crops to estimate, as a direct output of the aggregate trait estimation model, an aggregate trait value of the crops in the agricultural plot; and
cause data indicative of the estimated aggregate trait value of the crops in the agricultural plot to be output at a computing device.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, further comprising instructions to capture the plurality of images as a first plurality of images using a first vision sensor carried across the agricultural plot by an agricultural vehicle.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, further comprising instruction to capture a second plurality of images using a second vision sensor carried across the agricultural plot by the agricultural vehicle, wherein the first vision sensor captures the crops from a first angle and the second vision sensor captures the crops from a second angle that is different than the first angle.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, wherein the first angle comprises a side view of the crops and the second angle comprises a top-down view of the crops.
