# US 12,181,889 B2 · Method for controlling a vehicle for harvesting agricultural material > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,181,889 B2 - Document ID: US12181889B2 - Status: Granted - Kind: B2 - Application number: 18/047,519 - Filed: 2022-10-18 - Published: 2024-04-18 - Granted: 2024-12-31 - Earliest claimed: 2022-10-18 - Assignees: Deere & Company - Applicants: Deere & Company - Inventors: Ashraf Qadir; Qiang Ray Liu; Ryan R. White - Web page: https://patentgrep.com/patent/US12181889B2 ## CPC - G05D 1/0251: {extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision (stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593)} - A01B 79/005: {Precision agriculture} - A01D 41/127: Control or measuring arrangements specially adapted for combines - A01D 41/1271: {for measuring crop flow} - G05D 1/0278: {using satellite positioning signals, e.g. GPS} - G05D 1/2465: {using a 3D model of the environment} - G05D 2105/15: for harvesting, sowing or mowing in agriculture or forestry - G05D 2107/21: {Farming, e.g. fields, pastures or barns} - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21) - A01D: HARVESTING; MOWING ## Abstract An upper point cloud estimator is configured to estimate a three-dimensional representation of the crop canopy based on collected stereo vision image data. A lower point cloud estimator is configured to estimate a ground three-dimensional representation or lower point of the ground based on the determined average. The electronic data processor is configured to determine one or more differences between the upper point cloud (or upper surface) of the crop canopy and a lower point cloud of the ground, where each difference is associated with a cell within a grid defined by the front region. The electronic data processor is capable of providing the differences to a data processing system to estimate a yield or differential yield for the front region, among other things. ## Claims ### Claim 1 A method for controlling a vehicle for performing an agricultural task on an agricultural material in a field or at a worksite, the method comprising: collecting a stereo vision data to determine a crop three-dimensional representation or upper point cloud of an upper portion of a crop canopy in a front region in front of the vehicle; estimating an observed ground elevation data based on a current observed position data of a location-determining receiver, the current observed position data comprising an altitude data and an attitude data of the ground associated with the front region; accessing a stored ground elevation data based on a prior position data that aligns with the current observed position data, the prior position data comprising the altitude data and the attitude data of the ground; determining an average or weighted average of the observed ground elevation data and the stored ground elevation data for the aligned prior position data and the current observed position data; estimating a ground three-dimensional representation or lower point cloud of the ground based on the determined average; transforming or aligning the collected stereo vision data to a common coordinate system of the estimated lower point cloud of the ground; determining differences between the upper point cloud of the crop canopy and a lower point cloud of the ground, where each difference is associated with a cell within a grid defined by the front region; providing the differences to a data processing system to estimate a yield or differential yield for the front region; and adjusting a ground speed or a ground velocity of the vehicle when the yield or differential yield in one or more cells satisfies a threshold, or exceeds a lower yield limit, or is less than an upper yield limit. ### Claim 2 (depends on 1) The method according to claim 1, further comprising: estimating, by a location-determining receiver or global navigation satellite receiver on the vehicle or an implement mechanically coupled to the vehicle, prior ground elevation data and corresponding prior position data; and recording or storing, in a data storage device, the prior ground elevation data associated with corresponding prior position data of the vehicle or the implement mechanically coupled to the vehicle that is performing the agricultural task in the field or at the worksite. ### Claim 3 (depends on 2) The method according to claim 2, wherein the agricultural task comprises planting, seeding, tilling or spraying. ### Claim 4 (depends on 1) The method according to claim 1, further comprising: estimating, by a location-determining receiver or global navigation satellite receiver, prior ground elevation data and corresponding prior position data; determining, by a stereo vision imaging device coupled to the vehicle for performing tillage, planting or seeding, prior ground elevation data associated with the corresponding prior position data; and recording or storing, in a data storage device, the prior ground elevation data associated with corresponding prior position data of the vehicle or an implement mechanically coupled to the vehicle that is performing an agricultural task in the field or at the worksite. ### Claim 5 (depends on 4) The method according to claim 4, further comprising transforming or mapping an initial coordinate reference frame of the prior ground elevation data of the stereo imaging device to a common coordinate reference frame of the location-determining receiver. ### Claim 6 (depends on 1) The method according to claim 1, further comprising: estimating, by a location-determining receiver or global navigation satellite receiver, prior ground elevation data and corresponding prior position data; determining, by one or more range or distance sensors on a boom of the vehicle configured as a sprayer vehicle, prior ground elevation data associated with the corresponding prior position data; and recording or storing, in a data storage device, the prior ground elevation data associated with corresponding prior position data of the vehicle or an implement mechanically coupled to the vehicle that is performing an agricultural task in the field or at the worksite. ### Claim 7 (depends on 6) The method according to claim 6, further comprising transforming or mapping an initial coordinate reference frame of the prior ground elevation data of the range or distance sensors to a common coordinate reference frame of the location-determining receiver. ### Claim 8 (depends on 1) The method according to claim 1, wherein estimating the differential yield comprises determining the difference between the crop height and ground elevation to estimate yield or volume or mass of plant material to be harvested by the vehicle configured as a combine or harvester within one or more cells. ### Claim 9 A method for controlling a vehicle for performing an agricultural task on an agricultural material in a field or at a worksite, the method comprising: collecting a stereo vision data to determine a crop three-dimensional representation or upper point cloud of an upper portion of a crop canopy in a front region in front of the vehicle; estimating an observed ground elevation data based on a current observed position data of a location-determining receiver, the current observed position data comprising an attitude data of the ground associated with the front region; accessing a stored ground elevation data based on a prior position data that aligns with the current observed position data, the prior position data comprising an altitude data and the attitude data of the ground that is averaged over time for the field or worksite; estimating a ground three-dimensional representation or lower point cloud of the ground based on the stored ground elevation data; transforming or aligning the collected stereo vision data to a common coordinate system of the estimated lower point cloud of the ground; determining differences between the upper point cloud of the crop canopy and a lower point cloud of the ground, where each difference is associated with a cell within a grid defined by the front region; providing the differences to a data processing system to estimate a yield or differential yield for the front region; and adjusting a ground speed or a ground velocity of the vehicle when the yield or differential yield in one or more cells satisfies a threshold, or exceeds a lower yield limit, or is less than an upper yield limit. ### Claim 10 (depends on 9) The method according to claim 9, further comprising determining differential settings of vehicle speed or velocity related to harvesting of the crop associated with the front region. ### Claim 11 (depends on 9) The method according to claim 9, further comprising adjusting the speed or velocity of the vehicle to maintain a target throughput or output of estimated ground elevation. ### Claim 12 (depends on 9) The method according to claim 9, wherein the attitude data comprises roll angle data associated with the vehicle, and wherein the lower point cloud of the ground is adjusted to account for the roll data over a vehicle width or swath or for a roll-adjusted ground region between a plurality of vehicle wheels or tracks. ### Claim 13 (depends on 9) The method according to claim 9, wherein the attitude data comprises roll angle, pitch angle, and heading angle data associated with the vehicle and wherein the lower point cloud of the ground is adjusted to account for the pitch angle data for a pitch-adjusted ground region between a front wheel and a rear wheel or between a front and a rear vehicle tracks or a pitch-adjusted ground region between a vehicle front and a vehicle rear is adjusted to account for the pitch angle. ### Claim 14 (depends on 9) The method according to claim 9, wherein an upper boundary of an upper cell of the upper point cloud is projected downward to a corresponding lower boundary of a lower cell of the lower point cloud or where the lower boundary of the lower cell of the lower point cloud is projected upward to the corresponding upper boundary of the upper cell of the upper point cloud. ### Claim 15 (depends on 14) The method according to claim 14, wherein points of the point cloud lying on the upper boundary or the lower boundary have pairs of equal x-y plane coordinates, with different z coordinates. ### Claim 16 (depends on 14) The method according to claim 14, wherein a corresponding three dimensional volume of an upper-lower cell pair between the upper boundary and the lower boundary is indicative of the yield or vigor of plants or crops in the cell. ### Claim 17 (depends on 16) The method according to claim 16, wherein an aggregate yield of cells within the front region is used to adjust a vehicle setting, and to calibrate or supplement a yield detected by the yield monitoring during a time interval. ### Claim 18 (depends on 16) The method according to claim 16, wherein an aggregate yield of cells within an entire field region is used to estimate the yield of the crop, alone or together with observed measurements of a yield monitor. ### Claim 19 (depends on 14) The method according to claim 14, wherein an average, mean, median or mode of a difference between an upper height for a corresponding upper cell and a lower height for a lower cell is indicative of the yield or vigor of plants or crops in the cell. ### Claim 20 (depends on 9) The method according to claim 9, wherein one or more points or cells within the lower point cloud of the ground are adjusted or corrected based on previously stored elevation surveys of the field. ### Claim 21 (depends on 9) The method according to claim 9, wherein the location-determining receiver comprises a global navigation satellite system and a wireless device that operates in a real-time kinematic (RTK) mode and that receives a correction signal from a local or applicable RTK reference station. ### Claim 22 (depends on 9) The method according to claim 9, wherein the location-determining receiver comprised a global navigation satellite system and a wireless device that operates in a precise point positioning (PPP) mode and that receives a correction signal from a central data processing system associated with a network of satellite receiver reference stations. ### Claim 23 (depends on 9) The method according to claim 9, further comprising a radar or LIDAR device that is directed toward the front region of the vehicle to provide potential adjustments or augmentation of the elevation data versus position or geographic coordinates of the lower point cloud of the ground.