# US 12,008,730 B2 · Cloud-based framework for processing, analyzing, and visualizing imaging data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,008,730 B2
- Document ID: US12008730B2
- Status: Granted
- Kind: B2
- Application number: 18/328,897
- Filed: 2023-06-05
- Published: 2023-10-12
- Granted: 2024-06-11
- Earliest claimed: 2020-04-22
- Assignees: University of Florida Research Foundation, Inc.
- Applicants: University of Florida Research Foundation, Incorporated
- Inventors: Ioannis Ampatzidis; Victor H. Meirelles Partel; Lucas Fideles Costa
- Web page: https://patentgrep.com/patent/US12008730B2

## CPC

- G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images
- G06F 18/24133: {Distances to prototypes}
- G06F 18/40: Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/38: Registration of image sequences
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)}
- G06V 10/16: {using multiple overlapping images; Image stitching}
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/82: using neural networks
- G06V 10/94: Hardware or software architectures specially adapted for image or video understanding
- G06V 10/945: {User interactive design; Environments; Toolboxes}
- G06V 10/95: {structured as a network, e.g. client-server architectures}
- G06V 10/96: Management of image or video recognition tasks
- G06V 20/17: taken from planes or by drones
- G06V 20/188: {Vegetation}
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2200/28: involving image processing hardware
- G06T 2207/10024: Color image
- G06T 2207/10036: Multispectral image; Hyperspectral image
- G06T 2207/10048: Infrared image
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20104: Interactive definition of region of interest [ROI]
- G06T 2207/30188: Vegetation; Agriculture
- G06V 2201/07: Target detection
- Y02A 40/10: in agriculture
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- Y02A: TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE

## Abstract

Embodiments of the present disclosure provide methods, apparatus, systems, computing devices, computing entities, and/or the like for detecting objects located in an area of interest. In accordance with one embodiment, a method is provided comprising: receiving, via an interface provided through a general instance on a cloud environment, imaging data comprising raw images collected on the area of interest; upon receiving the images: activating a central processing unit (CPU) focused instance on the cloud environment and processing, via the image, the raw images to generate an image map of the area of interest; and after generating the image map: activating a graphical processing unit (GPU) focused instance on the cloud environment and performing object detection, via the image, on a region within the image map by applying one or more object detection algorithms to the region to identify locations of the objects in the region.

## Claims

### Claim 1

A method for detecting a plurality of objects for an area of interest, the method comprising:
receiving, by a general instance on a cloud environment, imaging data comprising a plurality of raw images collected on the area of interest;
generating, by a central processing unit (CPU)-focused instance on the cloud environment, an image map of the area of interest by stitching together at least two raw images of the plurality of raw images to generate the image map of the area of interest; activating, by the general instance, a graphical processing unit
(GPU)-focused instance on the cloud environment based at least in part on a GPU-focused machine configuration image;
performing object detection, by the GPU-focused instance on at least a region within the image map by applying one or more object detection algorithms to the region of the image map to identify locations of the plurality of objects in the region of the image map;
closing, by the general instance, the GPU-focused instance on the cloud environment; and
initiating performance of one or more actions based at least in part on the plurality of objects detected for the region within the image map.

### Claim 2 (depends on 1)

The method of claim 1, wherein performing the object detection on the region within the image map comprises:
preprocessing the image map to reduce variation in the image map resulting from capturing the plurality of raw images;
applying a first object detection algorithm of the one or more object detection algorithms to identify initial locations of the plurality of objects in the region of the image map;
identifying one or more object patterns in the image map;
analyzing the one or more object patterns to identify one or more false positives in the initial locations of the plurality of objects;
removing the one or more false positives from the initial locations of the plurality of objects; and
after removing the one or more false positives from the initial locations of the plurality of objects, applying a second object detection algorithm of the one or more object detection algorithms to each of the one or more object patterns to identify the locations of the plurality of objects.

### Claim 3 (depends on 2)

The method of claim 2, wherein the area of interest comprises a tree grove, the plurality of objects comprises a plurality of trees, and the one or more object patterns comprise one or more rows of trees.

### Claim 4 (depends on 1)

The method of claim 1 further comprising receiving input originating from a user, wherein the input comprises at least one of the region within the image map, an object spacing identifying an average space between objects located in the region, or one or more blank areas found in the region to be skipped by the one or more object detection algorithms.

### Claim 5 (depends on 1)

The method of claim 1, wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more actions comprise:
generating processing data for the plurality of trees, the processing data comprising at least one of a total number of trees for the plurality of trees, one or more tree gap counts, an average value of tree heights, a tree height for one or more of the plurality of trees, a canopy area estimation for one or more of the plurality of trees, a yield estimation of fruit for one or more of the plurality of trees, an estimation of tree ages for one or more of the plurality of trees, an estimation of tree health for one or more of the plurality of trees, estimated nutrient concentrations for one or more of the plurality of trees, or a fertility map based at least in part on estimated nutrient concentrations for the plurality of trees; and
providing at least a portion of the processing data for display via an interface on a user device.

### Claim 6 (depends on 5)

The method of claim 5, wherein the one or more actions comprise:
receiving an input indicating a selection of the region from a user via the user device; and
responsive to receiving the input indicating the selection of the region, providing a map of the region for display via the interface on the user device, the map displaying the plurality of trees detected for the region.

### Claim 7 (depends on 1)

The method of claim 1, wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more actions comprise: generating an application map identifying an application rate for at least the region within the image map; and
providing the application map to a smart sprayer system configured to use the application map to control flow of a liquid being applied to the plurality of trees in the region.

### Claim 8

A system for detecting a plurality of objects for an area of interest, the system comprising at least one processor and at least one memory including program code, the at least one memory and the program code configured to, with the at least one processor, cause the system to at least:
receive, by a general instance on a cloud environment, imaging data comprising a plurality of raw images collected on the area of interest;
generate, by a central processing unit (CPU)-focused instance on the cloud environment, an image map of the area of interest by stitching together at least two raw images of the plurality of raw images to generate the image map of the area of interest; activate, by the general instance, a graphical processing unit
(GPU)-focused instance on the cloud environment based at least in part on a GPU-focused machine configuration image;
perform object detection, by the GPU-focused instance, on at least a region within the image map by applying one or more object detection algorithms to the region of the image map to identify locations of the plurality of objects in the region of the image map;
close, by the general instance, the GPU-focused instance on the cloud environment; and
initiate performance of one or more actions based at least in part on the plurality of objects detected for the region within the image map.

### Claim 9 (depends on 8)

The system of claim 8, wherein the GPU-focused instance performs the object detection on the region within the image map by:
preprocessing the image map to reduce variation in the image map resulting from capturing the plurality of raw images;
applying a first object detection algorithm of the one or more object detection algorithms to identify initial locations of the plurality of objects in the region of the image map;
identifying one or more object patterns in the image map;
analyzing the one or more object patterns to identify one or more false positives in the initial locations of the plurality of objects;
removing the one or more false positives from the initial locations of the plurality of objects; and
after removing the one or more false positives from the initial locations of the plurality of objects, applying a second object detection algorithm of the one or more object detection algorithms to each of the one or object patterns to identify the locations of the plurality of objects.

### Claim 10 (depends on 9)

The system of claim 9, wherein the area of interest comprises a tree grove, the plurality of objects comprises a plurality of trees, and the one or more object patterns comprise one or more rows of trees.

### Claim 11 (depends on 8)

The system of claim 8, wherein the GPU-focused instance performs the object detection of the region of the image map by receiving input originating from a user, the input comprising at least one of the region within the image map, an object spacing identifying an average space between objects located in the region, or one or more blank areas found in the region to be skipped by the one or more object detection algorithms.

### Claim 12 (depends on 8)

The system of claim 8, wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more actions comprise:
generating processing data for the plurality of trees, the processing data comprising at least one of a total number of trees for the plurality of trees, one or more tree gap counts, an average value of tree heights, a tree height for one or more of the plurality of trees, a canopy area estimation for one or more of the plurality of trees, a yield estimation of fruit for one or more of the plurality of trees, an estimation of tree ages for one or more of the plurality of trees, an estimation of tree health for one or more of the plurality of trees, estimated nutrient concentrations for one or more of the plurality of trees, or a fertility map based at least in part on estimated nutrient concentrations for the plurality of trees; and
providing at least a portion of the processing data for display via an interface on a user device.

### Claim 13 (depends on 12)

The system of claim 12, wherein the one or more actions comprise:
receiving an input indicating a selection of the region from a user via the user device; and responsive to receiving the input indicating the selection of the region, providing a map of the region for display via the interface on the user device, the map displaying the plurality of trees detected for the region.

### Claim 14 (depends on 8)

The system of claim 8, wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more actions comprise:
generating an application map identifying an application rate for at least the region within the image map; and
providing the application map to a smart sprayer system configured to use the application map to control flow of a liquid being applied to the plurality of trees in the region.

### Claim 15

A non-transitory computer storage medium comprising instructions for detecting a plurality of objects for an area of interest, the instructions being configured to cause one or more processors to at least perform operations configured to:
receive, by a general instance on a cloud environment, imaging data comprising a plurality of raw images collected on the area of interest;
generate, by a central processing unit (CPU)-focused instance on the cloud environment, an image map of the area of interest by stitching together at least two raw images of the plurality of raw images to generate the image map of the area of interest; activate, by the general instance, a graphical processing unit
(GPU)-focused instance on the cloud environment based at least in part on a GPU-focused machine configuration image;
perform object detection, by the GPU-focused instance, on at least a region within the image map by applying one or more object detection algorithms to the region of the image map to identify locations of the plurality of objects in the region of the image map;
close, by the general instance, the GPU-focused instance on the cloud environment; and
initiate performance of one or more actions based at least in part on the plurality of objects detected for the region within the image map.

### Claim 16 (depends on 15)

The non-transitory computer storage medium of claim 15, wherein the GPU-focused instance performs the object detection on the region within the image map by:
preprocessing the image map to reduce variation in the image map resulting from capturing the plurality of raw images;
applying a first object detection algorithm of the one or more object detection algorithms to identify initial locations of the plurality of objects in the region of the image map;
identifying one or more object patterns in the image map;
analyzing the one or more object patterns to identify one or more false positives in the initial locations of the plurality of objects;
removing the one or more false positives from the initial locations of the plurality of objects; and
after removing the one or more false positives from the initial locations of the plurality of objects, applying a second object detection algorithm of the one or more object detection algorithms to each of the one or more object patterns to identify the locations of the plurality of objects.

### Claim 17 (depends on 16)

The non-transitory computer storage medium of claim 16, wherein the area of interest comprises a tree grove, the plurality of objects comprises a plurality of trees, and the one or more object patterns comprise one or more rows of trees.

### Claim 18 (depends on 15)

The non-transitory computer storage medium of claim 15, wherein the GPU-focused instance performs the object detection of the region of the image map by receiving input originating from a user, the input comprising at least one of the region within the image map, an object spacing identifying an average space between objects located in the region, or one or more blank areas found in the region to be skipped by the one or more object detection algorithms.

### Claim 19 (depends on 15)

The non-transitory computer storage medium of claim 15, wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more actions comprise:
generating processing data for the plurality of trees, the processing data comprising at least one of a total number of trees for the plurality of trees, one or more tree gap counts, an average value of tree heights, a tree height for one or more of the plurality of trees, a canopy area estimation for one or more of the plurality of trees, a yield estimation of fruit for one or more of the plurality of trees, an estimation of tree ages for one or more of the plurality of trees, an estimation of tree health for one or more of the plurality of trees, estimated nutrient concentrations for one or more of the plurality of trees, or a fertility map based at least in part on estimated nutrient concentrations for the plurality of trees; and
providing at least a portion of the processing data for display via an interface on a user device.

### Claim 20 (depends on 19)

The non-transitory computer storage medium of claim 19, wherein the one or more actions comprise:
receiving an input indicating a selection of the region from a user via the user device; and responsive to receiving the input indicating the selection of the region, providing a map of the region for display via the interface on the user device, the map displaying the plurality of trees detected for the region.

### Claim 21 (depends on 15)

The non-transitory computer storage medium of claim 15, wherein the area of interest comprises a tree grove, the plurality of objects comprise a plurality of trees, and the one or more actions comprise:
generating an application map identifying an application rate for at least the region within the image map; and
providing the application map to a smart sprayer system configured to use the application map to control flow of a liquid being applied to the plurality of trees in the region.