# US 12,117,831 B2 · Adaptive cyber-physical system for efficient monitoring of unstructured environments > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,117,831 B2 - Document ID: US12117831B2 - Status: Granted - Kind: B2 - Application number: 18/342,270 - Filed: 2023-06-27 - Published: 2024-04-18 - Granted: 2024-10-15 - Earliest claimed: 2017-01-23 - Assignees: The Board of Trustees of the University of Illinois - Applicants: The Board of Trustees of the University of Illinois - Inventors: Girish Chowdhary; Chinmay P. Soman; Beau David Barber - Web page: https://patentgrep.com/patent/US12117831B2 ## CPC - G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)} - G05D 1/2247: {providing the operator with simple or augmented images from one or more cameras} - G06F 16/29: Geographical information databases - G06F 18/2178: {based on feedback of a supervisor} - G06F 18/2321: using statistics or function optimisation, e.g. modelling of probability density functions - G06V 10/22: by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition - G06V 10/763: {Non-hierarchical techniques, e.g. based on statistics of modelling distributions} - G06V 10/7784: {based on feedback from supervisors} - G06V 20/17: taken from planes or by drones - G06V 20/188: {Vegetation} - G08G 5/34: for flight plan modification - B64C 39/024: {of the remote controlled vehicle type, i.e. RPV} - B64U 2101/30: for imaging, photography or videography - B64U 2101/40: for agriculture or forestry operations - G06V 20/194: {using hyperspectral data, i.e. more or other wavelengths than RGB} - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - B64C: AEROPLANES; HELICOPTERS - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR ## Abstract The present disclosure provides a system for monitoring unstructured environments. A predetermined path can be determined according to an assignment of geolocations to one or more agronomically anomalous target areas, where the one or more agronomically anomalous target areas are determined according to an analysis of a plurality of first images that automatically identifies a target area that deviates from a determination of an average of the plurality of first images that represents an anomalous place within a predetermined area, where the plurality of first images of the predetermined area are captured by a camera during a flight over the predetermined area. A camera of an unmanned vehicle can capture at least one second image of the one or more agronomically anomalous target areas as the unmanned vehicle travels along the predetermined path. ## Claims ### Claim 1 A system for monitoring environments, the system comprising: a base station comprising a first processing system including at least one first processor; and an unmanned aerial vehicle (UAV) comprising a second processing system including at least one second processor, wherein the UAV is in communication with the base station, wherein the second processing system of the UAV is configured to capture a plurality of initial images of a predetermined area to obtain coverage of the predetermined area at a lower spatial resolution as the UAV flies along a flight path, wherein the flight path for the UAV is above the predetermined area, and wherein the second processing system of the UAV is configured to transmit the plurality of initial images to the first processing system of the base station, wherein the first processing system of the base station is configured to analyze the at least one of the plurality of initial images to automatically identify areas within the at least one of the plurality of initial images that deviate from a determination of an average of the plurality of initial images and that represent anomalous places within the predetermined area to identify a plurality of agronomically anomalous target areas, and wherein the plurality of agronomically anomalous target areas together comprise a set of agronomically anomalous target areas, wherein the first processing system of the base station is configured to select one or more of the agronomically anomalous target areas of the set to form a subset of one or more agronomically anomalous target areas, wherein a number of agronomically anomalous target areas in the set is larger than is in the subset, wherein each agronomically anomalous target area in the subset is selected by the first processing system of the base station as a result of that agronomically anomalous target area being larger than at least one agronomically anomalous target area of the set that is excluded from the subset, and wherein the first processing system of the base station is configured to transmit an updated flight path to the second processing system of the UAV directing the UAV to the one or more agronomically anomalous target areas of the subset, and wherein the second processing system of the UAV is configured to, in response to receiving identification of the one or more agronomically anomalous target areas of the subset, automatically cause the UAV to capture one or more additional high resolution images of the one or more agronomically anomalous target areas of the subset. ### Claim 2 (depends on 1) The system of claim 1, wherein the first processing system of the base station is configured to examine the at least one of the plurality of initial images that has been transformed to an alternative colorspace to identify locations that have different spectral signatures from the average. ### Claim 3 (depends on 2) The system of claim 2, wherein the at least one of the plurality of initial images has been transformed from an RGB colorspace to the alternative colorspace, and wherein the alternative colorspace is selected from hue-saturation-value (HSV) or multispectral. ### Claim 4 (depends on 3) The system of claim 3, wherein the first processing system of the base station is configured to perform a grid search of blocks of images in the alternative colorspace. ### Claim 5 A non-transitory computer-readable storage medium comprising executable instructions that, when executed by a processing system including at least one processor, perform operations comprising: analyzing at least one initial image of a plurality of first images of an area to automatically identify a plurality of target areas, each of which deviates from a determination of an average of the plurality of first images and each of which represents an anomalous place within the area that is identified as an agronomically anomalous target area, the agronomically anomalous target areas collectively forming a set of agronomically anomalous target areas, and wherein the plurality of first images of the area are captured at a first spatial resolution; selecting one or more of the agronomically anomalous target areas of the set to form a subset of one or more agronomically anomalous target areas, wherein a number of agronomically anomalous target areas in the set is larger than is in the subset, wherein each agronomically anomalous target area in the subset is selected as a result of that agronomically anomalous target area being larger than at least one agronomically anomalous target area of the set that is excluded from the subset; and obtaining at least one second image of the one or more agronomically anomalous target areas of the subset that is captured by a camera of an unmanned vehicle as the unmanned vehicle moves along a path, wherein the path is generated according to the selecting of the subset, wherein the path directs the unmanned vehicle to the one or more agronomically anomalous target areas of the subset, and wherein the at least one second image is at a second spatial resolution which is higher than the first spatial resolution. ### Claim 6 (depends on 5) The non-transitory computer-readable storage medium of claim 5, wherein the processing system is located on the unmanned vehicle. ### Claim 7 (depends on 5) The non-transitory computer-readable storage medium of claim 5, wherein the unmanned vehicle is an unmanned aerial vehicle (UAV), wherein the processing system is located on a mobile device that is in wireless communication with the UAV, wherein the processing system wirelessly transmits an updated flight path to the UAV, wherein the plurality of first images and the at least one second image are captured by the camera during a single flight of the UAV, and wherein the obtaining of the plurality of first images and the at least one second image by the processing system is via wireless communications with the UAV. ### Claim 8 (depends on 5) The non-transitory computer-readable storage medium of claim 5, wherein the analyzing comprises transforming the at least one initial image of the plurality of first images from an RGB colorspace to an alternative colorspace. ### Claim 9 (depends on 8) The non-transitory computer-readable storage medium of claim 8, wherein the alternative colorspace is selected from hue-saturation-value (HSV) or multispectral. ### Claim 10 (depends on 8) The non-transitory computer-readable storage medium of claim 8, wherein the operations further comprise performing a grid search of blocks of images in the alternative colorspace. ### Claim 11 (depends on 5) The non-transitory computer-readable storage medium of claim 5, wherein the unmanned vehicle comprises an unmanned aerial vehicle. ### Claim 12 (depends on 5) The non-transitory computer-readable storage medium of claim 5, wherein the unmanned vehicle comprises an unmanned ground vehicle. ### Claim 13 A method comprising: obtaining, by a processing system including at least one processor of an unmanned vehicle, a predetermined path, wherein the predetermined path is determined according to: an analysis of at least one initial image of a plurality of first images to automatically identify a plurality of target areas, each of which deviates from a determination of an average of the plurality of first images and each of which identifies an agronomically anomalous target area, wherein the agronomically anomalous target areas collectively form a set of agronomically anomalous target areas, and wherein the plurality of first images are captured at a first spatial resolution; and a selection of one or more of the agronomically anomalous target areas of the set to form a subset of one or more agronomically anomalous target areas, wherein a number of agronomically anomalous target areas in the set is larger than is in the subset, wherein each agronomically anomalous target area in the subset is selected as a result of that agronomically anomalous target area being larger than at least one agronomically anomalous target area of the set that is excluded from the subset; and capturing, by the processing system via a camera of the unmanned vehicle, at least one second image of the one or more agronomically anomalous target areas of the subset as the unmanned vehicle is caused to travel along the predetermined path, wherein the at least one second image is at a second spatial resolution which is higher than the first spatial resolution; wherein the analysis of the at least one initial image comprises transforming the at least one initial image from an RGB colorspace to an alternative colorspace; and wherein machine learning algorithms are trained to automatically identify specific causes of anomalies. ### Claim 14 (depends on 13) The method of claim 13, wherein the alternative colorspace is selected from hue-saturation-value (HSV) or multispectral. ### Claim 15 (depends on 13) The method of claim 13, wherein the unmanned vehicle is a ground vehicle, the plurality of first images are captured by an unmanned aerial vehicle, and the plurality of first images are captured by the unmanned aerial vehicle during a flight. ### Claim 16 (depends on 13) The method of claim 13, wherein the unmanned vehicle is a low-flying unmanned aerial vehicle, the plurality of first images are captured by a high-flying unmanned aerial vehicle, and the plurality of first images are captured by the high-flying unmanned aerial vehicle during a flight. ### Claim 17 (depends on 13) The method of claim 13, wherein the predetermined path is determined by a mobile device in wireless communication with the processor of the unmanned vehicle, and wherein the one or more agronomically anomalous target areas of the subset are determined by the mobile device. ### Claim 18 (depends on 13) The method of claim 13, wherein the at least one second image is overlayed at an appropriate geolocation and displayed to a user. ### Claim 19 (depends on 13) The method of claim 13, wherein: feedback is collected from a user; and the feedback is associated with the plurality of first images, the at least one second image, or a combination thereof. ### Claim 20 (depends on 19) The method of claim 19, wherein the feedback from the user is selected from text, audio, thumbs up or thumbs down, or a combination thereof.