# US 2019/0166752 A1 · Monitoring Aerial Application Tasks and Recommending Corrective Actions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0166752 A1 - Document ID: US20190166752A1 - Status: Published application - Kind: A1 - Application number: 15/828,575 - Filed: 2017-12-01 - Published: 2019-06-06 - Earliest claimed: 2017-12-01 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Stefano Sidoti; Agostino Sturaro - Web page: https://patentgrep.com/patent/US20190166752A1 ## CPC - G05D 1/0094: {involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target} - A01B 79/005: {Precision agriculture} - B64D 1/16: Dropping or releasing powdered, liquid, or gaseous matter, e.g. for fire-fighting (jettisoning fuel B64D37/26) - G05D 1/104: {involving a plurality of aircrafts, e.g. formation flying (traffic control systems for aircraft G08G5/00)} - G06T 5/80: Geometric correction - G06V 20/188: {Vegetation} - B64U 2101/20: for use as communications relays, e.g. high-altitude platforms - B64U 2101/30: for imaging, photography or videography - G06T 11/60: Creating or editing images; Combining images with text - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21) - B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR ## Abstract Recommending corrective action during aerial application is provided. An unmanned aerial vehicle is navigated to a geolocation where an aerial application task is currently carried out. First sensor data is received from the unmanned aerial vehicle that characterize a quality of the aerial application task. Measures to be carried out to increase the quality of the aerial application task are determined based on the first sensor data. The measures are outputted while the aerial application task is ongoing so that the aerial application task can be adapted while the aerial application task occurs. ## Claims ### Claim 1 A method for recommending corrective action during aerial application, the method comprising: navigating an unmanned aerial vehicle to a geolocation where an aerial application task is currently carried out; receiving first sensor data from the unmanned aerial vehicle that characterize a quality of the aerial application task; determining measures to be carried out to increase the quality of the aerial application task based on the first sensor data; and outputting the measures while the aerial application task is ongoing so that the aerial application task can be adapted while the aerial application task occurs. ### Claim 2 (depends on 1) The method of claim 1 further comprising: receiving second sensor data from a second aerial vehicle performing the aerial application task that characterize properties corresponding to the second aerial vehicle, wherein the determining of the measures is based on the first sensor data and the second sensor data. ### Claim 3 (depends on 2) The method of claim 2, wherein the second aerial vehicle is an artificial intelligence controlled aerial vehicle, the method further comprising: calculating control commands based on the measures; and sending the control commands to the artificial intelligence controlled aerial vehicle to adapt the aerial application task while the aerial application task is ongoing. ### Claim 4 (depends on 3) The method of claim 3, wherein the second aerial vehicle is a human controlled aerial vehicle, the method further comprising: generating a recommendation with instructions on how to adapt the aerial application task based on the control commands; and sending the recommendation on how to adapt the aerial application task to the human controlled aerial vehicle while the aerial application task is ongoing. ### Claim 5 (depends on 1) The method of claim 1 further comprising: generating a lookahead tree having a root node that represents a current situation in a timeline corresponding to the aerial application task; setting a first timer for a first computation phase, which estimates a level of risk corresponding to the aerial application task, to a first computation time threshold limit; setting a first prediction limit for the first computation phase that defines a maximum number of defined time periods to predict into a future; setting a random walk node selection strategy to select nodes in the lookahead tree; selecting a leaf node in the lookahead tree utilizing the set node selection strategy; setting the leaf node as a new current situation; and adding a path from the root node to the leaf node to the timeline to obtain a hypothetical timeline of what occurred up to the new current situation. ### Claim 6 (depends on 5) The method of claim 5 further comprising: selecting a pilot decision for the new current situation that an agricultural aircraft pilot may take while performing the aerial application task based on previous pilot decisions taken during same or similar situations; generating a forecast of a weather change that can occur during the new current situation; applying the pilot decision and the weather change to the current situation one defined time period into the future; predicting a consequence of the new current situation that is a result of the pilot decision and the weather change applied to the new current situation one defined time period into the future; and adding the consequence of the new current situation to the lookahead tree. ### Claim 7 (depends on 6) The method of claim 6 further comprising: selecting a predicted consequence of the new current situation from the lookahead tree using the set node selection strategy; setting the predicted consequence as the new current situation; running a simulation of possible series of events occurring from the predicted consequence of the new current situation that extends the hypothetical timeline by applying pilot decisions, weather changes, and situation consequences recursively up to the first prediction limit for the first computation phase; and utilizing a last situation reached in the hypothetical timeline at the first prediction limit corresponding to the first computation phase as a situation simulation. ### Claim 8 (depends on 7) The method of claim 7 further comprising: generating a score indicating how desirable the situation simulation is for the aerial application task; and updating scores of the predicted consequence and its ancestor nodes in the lookahead tree to account for the score corresponding to the situation simulation reflecting an impact of a possible outcome of the situation simulation. ### Claim 9 (depends on 8) The method of claim 8 further comprising: determining whether the first timer reached the first computation time threshold limit; responsive to determining that the first timer has reached the first computation time threshold limit, determining whether a second computation phase for generating corrective action recommendations has already started; responsive to determining that the second computation phase has not already started, evaluating the level of risk corresponding to the current situation at the root node of the lookahead tree taking into account what occurred before the new current situation in the timeline and what could happen next in the lookahead tree considering all possible consequences; determining whether the level of risk is greater than or equal to a risk level threshold value; and responsive to determining that the level of risk is greater than or equal to the risk level threshold value, starting the second computation phase. ### Claim 10 (depends on 9) The method of claim 9 further comprising: setting a second timer for the second computation phase to a second computation time threshold limit; setting a second prediction limit, which defines a maximum number of defined time periods to predict into the future, for the second computation phase; starting the second timer for the second computation phase; and setting an Upper Confidence Bound 1 selection strategy to select nodes in the lookahead tree. ### Claim 11 (depends on 10) The method of claim 10 further comprising: determining whether the second timer reached the second computation time threshold limit; and responsive to determining that the second timer has reached the second computation time threshold limit, selecting a corrective action recommendation for the agricultural aircraft pilot that has a best outcome in the lookahead tree of decreasing the level of risk. ### Claim 12 (depends on 1) The method of claim 1, wherein the unmanned aerial vehicle is a dedicated monitoring unmanned aerial vehicle that captures images of a second aerial vehicle performing the aerial application task, a target area of the aerial application task, and a cloud of product emitted by the second aerial vehicle while performing the aerial application task. ### Claim 13 (depends on 12) The method of claim 12, wherein a remote computer controls the dedicated monitoring unmanned aerial vehicle. ### Claim 14 A computer system for recommending corrective action during aerial application, the computer system comprising: a bus system; a storage device connected to the bus system, wherein the storage device stores program instructions; and a processor connected to the bus system, wherein the processor executes the program instructions to: navigate an unmanned aerial vehicle to a geolocation where an aerial application task is currently carried out; receive first sensor data from the unmanned aerial vehicle that characterize a quality of the aerial application task; determine measures to be carried out to increase the quality of the aerial application task based on the first sensor data; and output the measures while the aerial application task is ongoing so that the aerial application task can be adapted while the aerial application task occurs. ### Claim 15 (depends on 14) The computer system of claim 14, wherein the processor further executes the program instructions to: receive second sensor data from a second aerial vehicle performing the aerial application task that characterize properties corresponding to the second aerial vehicle, wherein determining the measures is based on the first sensor data and the second sensor data. ### Claim 16 (depends on 15) The computer system of claim 15, wherein the second aerial vehicle is an artificial intelligence controlled aerial vehicle, and wherein the processor further executes the program instructions to: calculate control commands based on the measures; and send the control commands to the artificial intelligence controlled aerial vehicle to adapt the aerial application task while the aerial application task is ongoing. ### Claim 17 A computer program product for recommending corrective action during aerial application, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising: navigating an unmanned aerial vehicle to a geolocation where an aerial application task is currently carried out; receiving first sensor data from the unmanned aerial vehicle that characterize a quality of the aerial application task; determining measures to be carried out to increase the quality of the aerial application task based on the first sensor data; and outputting the measures while the aerial application task is ongoing so that the aerial application task can be adapted while the aerial application task occurs. ### Claim 18 (depends on 17) The computer program product of claim 17 further comprising: receiving second sensor data from a second aerial vehicle performing the aerial application task that characterize properties corresponding to the second aerial vehicle, wherein the determining of the measures is based on the first sensor data and the second sensor data. ### Claim 19 (depends on 18) The computer program product of claim 18, wherein the second aerial vehicle is an artificial intelligence controlled aerial vehicle, the method further comprising: calculating control commands based on the measures; and sending the control commands to the artificial intelligence controlled aerial vehicle to adapt the aerial application task while the aerial application task is ongoing. ### Claim 20 (depends on 18) The computer program product of claim 18, wherein the second aerial vehicle is a human controlled aerial vehicle, the method further comprising: generating a recommendation with instructions on how to adapt the aerial application task based on the control commands; and sending the recommendation on how to adapt the aerial application task to the human controlled aerial vehicle while the aerial application task is ongoing.