# US 12,111,888 B2 · Localization of individual plants based on high-elevation imagery > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,111,888 B2 - Document ID: US12111888B2 - Status: Granted - Kind: B2 - Application number: 17/344,328 - Filed: 2021-06-10 - Published: 2022-12-15 - Granted: 2024-10-08 - Earliest claimed: 2021-06-10 - Assignees: Deere & Company - Applicants: Deere & Company - Inventors: Zhiqiang Yuan - Web page: https://patentgrep.com/patent/US12111888B2 ## CPC - G06F 18/24: Classification techniques - G06V 10/82: using neural networks - B64C 39/024: {of the remote controlled vehicle type, i.e. RPV} - G06T 7/00: Image analysis - G06T 7/33: using feature-based methods - G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching} - G06V 20/182: {Network patterns, e.g. roads or rivers} - G06V 20/188: {Vegetation} - G06V 20/60: Type of objects - A01C 21/007: {Determining fertilization requirements} - A01D 46/30: Robotic devices for individually picking crops - A01M 21/00: Apparatus for the destruction of unwanted vegetation, e.g. weeds (control of undesirable vegetation on roads or permanent ways of railways E01H11/00) - B64U 2101/30: for imaging, photography or videography - B64U 2101/32: for cartography or topography - B64U 2101/40: for agriculture or forestry operations - G06T 2207/10032: Satellite or aerial image; Remote sensing - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30184: Infrastructure - G06T 2207/30188: Vegetation; Agriculture - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B64C: AEROPLANES; HELICOPTERS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A01C: PLANTING; SOWING; FERTILISING (parts, details or accessories of agricultural machines or implements, in general A01B51/00 - A01B75/00) - A01D: HARVESTING; MOWING - A01M: CATCHING, TRAPPING OR SCARING OF ANIMALS (appliances for catching swarms or drone-catching A01K57/00; fishing A01K69/00 - {A01K99/00}; biocides, pest repellants or attractants A01N); APPARATUS FOR THE DESTRUCTION OF NOXIOUS ANIMALS OR NOXIOUS PLANTS - B64U: UNMANNED AERIAL VEHICLES [UAV]; EQUIPMENT THEREFOR ## Abstract Implementations are described herein for localizing individual plants by aligning high-elevation images using invariant anchor points while disregarding variant feature points, such as deformable plants. High-elevation images that capture the plurality of plants at a resolution at which wind-triggered deformation of individual plants is perceptible between the high-elevation images may be obtained. First regions of the high-elevation images that depict the plurality of plants may be classified as variant features that are unusable as invariant anchor points. Second regions of the high-elevation images that are disjoint from the first set of regions may be classified as invariant anchor points. The high-elevation images may be aligned based on invariant anchor point(s) that are common among at least some of the high-elevation images. Based on the aligned high-elevation images, individual plant(s) may be localized within one of the high-elevation images for performance of one or more agricultural tasks. ## Claims ### Claim 1 A method for localizing one or more individual plants of a plurality of plants, the method implemented using one or more processors and comprising: obtaining high-elevation images that depict the plurality of plants at a resolution at which wind-triggered deformation of individual plants is perceptible between the high-elevation images; classifying a first set of regions of the high-elevation images that depict deformable plants as variant visual features that are unusable as invariant anchor points, wherein each region of the first set includes a respective cluster or bounded area of pixels that depict deformable plants; classifying a second set of regions of the high-elevation images that are disjoint from the first set of regions as invariant anchor points, wherein each region of the second set includes a respective cluster or bounded area of pixels; spatially aligning the high-elevation images based on one or more of the invariant anchor points that are common among at least some of the high-elevation images; and based on the spatially-aligned high-elevation images, localizing one or more individual plants of the plurality of plants within one of the high-elevation images for performance of one or more agricultural tasks. ### Claim 2 (depends on 1) The method of claim 1, wherein the localizing includes assigning position coordinates to the one or more individual plants based on position coordinates generated by an airborne vehicle that acquired the high-elevation images. ### Claim 3 (depends on 1) The method of claim 1, wherein the localizing includes mapping the one or more individual plants to one or more rows of a plurality of rows in which the plurality of plants are arranged. ### Claim 4 (depends on 3) The method of claim 3, further comprising processing the aligned high-elevation images to generate a map of the plurality of rows relative to the invariant anchor points. ### Claim 5 (depends on 1) The method of claim 1, wherein each region of the first and second sets of regions are pixel-wise classified. ### Claim 6 (depends on 1) The method of claim 1, further comprising deploying one or more agricultural robots to the one or more localized individual plants to perform one or more of the agricultural tasks. ### Claim 7 (depends on 1) The method of claim 1, wherein the high-elevation images are acquired by an unmanned aerial vehicle (UAV). ### Claim 8 (depends on 1) The method of claim 1, wherein classifying the first set of regions includes processing the high-elevation images based on one or more machine learning models that are trained to recognize one or more plant types. ### Claim 9 (depends on 1) The method of claim 1, wherein classifying the second set of regions includes detecting one or more lodged plants among the plurality of plants. ### Claim 10 (depends on 1) The method of claim 1, wherein classifying the second set of regions includes detecting agricultural equipment in spatial proximity with the plurality of plants. ### Claim 11 (depends on 1) The method of claim 1, wherein classifying the second set of regions includes detecting one or more water features or roads in spatial proximity with the plurality of plants. ### Claim 12 A system for localizing one or more individual plants of a plurality of plants, the system comprising one or more processors memory storing instructions that cause the one or more processors to: obtain high-elevation images that depict the plurality of plants at a resolution at which wind-triggered deformation of individual plants is perceptible between the high-elevation images; classify a first set of regions of the high-elevation images that depict deformable plants as variant visual features that are unusable as invariant anchor points, wherein each region of the first set includes a respective cluster or bounded area of pixels that depict deformable plants; classify a second set of regions of the high-elevation images that are disjoint from the first set of regions as invariant anchor points, wherein each region of the second set includes a respective cluster or bounded area of pixels; spatially align the high-elevation images based on one or more of the invariant anchor points that are common among at least some of the high-elevation images; and based on the spatially-aligned high-elevation images, localize one or more individual plants of the plurality of plants within one of the high-elevation images for performance of one or more agricultural tasks. ### Claim 13 (depends on 12) The system of claim 12, further comprising instructions to assign position coordinates to the one or more individual plants based on position coordinates generated by an airborne vehicle that acquired the high-elevation images. ### Claim 14 (depends on 12) The system of claim 12, further comprising instructions to map the one or more individual plants to one or more rows of a plurality of rows in which the plurality of plants are arranged. ### Claim 15 (depends on 14) The system of claim 14, further comprising instructions to process the aligned high-elevation images to generate a map of the plurality of rows relative to the invariant anchor points. ### Claim 16 (depends on 12) The system of claim 12, wherein each region of the first and second sets of regions are pixel-wise classified. ### Claim 17 (depends on 12) The system of claim 12, further comprising instructions to deploy one or more agricultural robots to the one or more localized individual plants to perform one or more of the agricultural tasks. ### Claim 18 (depends on 12) The system of claim 12, wherein the high-elevation images are acquired by an unmanned aerial vehicle (UAV). ### Claim 19 (depends on 12) The system of claim 12, further comprising instructions to process the high-elevation images based on one or more machine learning models that are trained to recognize one or more plant types. ### Claim 20 A non-transitory computer-readable medium for localizing one or more individual plants of a plurality of plants, the medium comprising instructions that, in response to execution of the instructions by a processor, cause the processor to: obtain high-elevation images that depict the plurality of plants at a resolution at which wind-triggered deformation of individual plants is perceptible between the high-elevation images; classify a first set of regions of the high-elevation images that depict deformable plants as variant visual features that are unusable as invariant anchor points, wherein each region of the first set includes a respective cluster or bounded area of pixels that depict deformable plants; classify a second set of regions of the high-elevation images that are disjoint from the first set of regions as invariant anchor points, wherein each region of the second set includes a respective cluster or bounded area of pixels; spatially align the high-elevation images based on one or more of the invariant anchor points that are common among at least some of the high-elevation images; and based on the spatially-aligned high-elevation images, localize one or more individual plants of the plurality of plants within one of the high-elevation images for performance of one or more agricultural tasks.