# US 12,112,501 B2 · Localization of individual plants based on high-elevation imagery > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,501 B2 - Document ID: US12112501B2 - Status: Granted - Kind: B2 - Application number: 17/354,147 - Filed: 2021-06-22 - Published: 2022-12-22 - Granted: 2024-10-08 - Earliest claimed: 2021-06-22 - Assignees: Deere & Company - Applicants: Deere & Company - Inventors: Zhiqiang Yuan; Jie Yang - Web page: https://patentgrep.com/patent/US12112501B2 ## CPC - G06T 7/74: {involving reference images or patches} - G06F 18/24: Classification techniques - G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting - G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images - G06T 7/33: using feature-based methods - G06V 10/24: Aligning, centring, orientation detection or correction of the image - G06V 10/82: using neural networks - G06V 20/182: {Network patterns, e.g. roads or rivers} - G06V 20/188: {Vegetation} - B25J 11/00: Manipulators not otherwise provided for - G06T 2207/10032: Satellite or aerial image; Remote sensing - G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform - G06T 2207/20081: Training; Learning - G06T 2207/30188: Vegetation; Agriculture - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) ## Abstract Implementations are described herein for localizing individual plants using high-elevation images at multiple different resolutions. A first set of high-elevation images that capture the plurality of plants at a first resolution may be analyzed to classify a set of pixels as invariant anchor points. High-elevation images of the first set may be aligned with each other based on the invariant anchor points that are common among at least some of the first set of high-elevation images. A mapping may be generated between pixels of the aligned high-elevation images of the first set and spatially-corresponding pixels of a second set of higher-resolution high-elevation images. Based at least in part on the mapping, individual plant(s) of the plurality of plants may be localized within one or more of the second set of high-elevation images for performance of one or more agricultural tasks. ## Claims ### Claim 1 A method for localizing one or more individual plants of a plurality of plants, the method implemented using one or more processors and comprising: analyzing a first set of high-elevation images that capture the plurality of plants at a first resolution; based on the analyzing, classifying a set of pixels of the first set of high-elevation images as depicting invariant anchor points, wherein the invariant anchor points include one or more visual features that are known to be movement-invariant; aligning high-elevation images of the first set based on one or more of the invariant anchor points that are common among at least some of the first set of high-elevation images; generating a mapping, based on the invariant anchor points, between pixels of the aligned high-elevation images of the first set and spatially-corresponding pixels of a second set of high-elevation images, wherein the second set of high-elevation images capture the plurality of plants at a second resolution that is greater than the first resolution; and based at least in part on the mapping, localizing one or more individual plants of the plurality of plants within one or more of the second set of high-elevation images for performance of one or more agricultural tasks. ### Claim 2 (depends on 1) The method of claim 1, further including downsampling the second set of high-elevation digital images to generate the first set of high-elevation digital images. ### Claim 3 (depends on 1) The method of claim 1, wherein the first set of high-elevation images are captured at a first elevation and the second set of high-elevation images are captured at a second elevation that is less than the first elevation. ### Claim 4 (depends on 1) The method of claim 1, wherein the localizing includes assigning position coordinates to the one or more individual plants based on position coordinates generated by an airborne vehicle that acquired the first set or the second set of high-elevation images. ### Claim 5 (depends on 1) The method of claim 1, wherein the localizing includes mapping the one or more individual plants to one or more rows of a plurality of rows in which the plurality of plants are arranged. ### Claim 6 (depends on 1) The method of claim 1, further including deploying one or more agricultural robots to the one or more localized individual plants to perform one or more of the agricultural tasks. ### Claim 7 (depends on 1) The method of claim 1, wherein the first set or the second set of high-elevation images is acquired by an unmanned aerial vehicle (UAV). ### Claim 8 (depends on 1) The method of claim 1, wherein the classifying includes processing the first set of high-elevation images based on one or more machine learning models that are trained to recognize one or more objects known to be movement-invariant. ### Claim 9 (depends on 8) The method of claim 8, wherein the classifying includes processing the first set of high-elevation images based on one or more machine learning models that are trained to recognize lodged plants among the plurality of plants. ### Claim 10 (depends on 8) The method of claim 8, wherein the classifying includes detecting agricultural equipment in spatial proximity with the plurality of plants. ### Claim 11 (depends on 8) The method of claim 8, wherein the classifying includes detecting one or more water features or roads in spatial proximity with the plurality of plants. ### Claim 12 A system for localizing one or more individual plants of a plurality of plants, the system comprising one or more processors and memory storing instructions that, in response to execution by the one or more processors, cause the one or more processors to: analyze a first set of high-elevation images that capture the plurality of plants at a first resolution; based on the analysis, classify a set of pixels of the first set of high-elevation images as depicting invariant anchor points, wherein the invariant anchor points include one or more visual features that are known to be movement-invariant; align high-elevation images of the first set based on one or more of the invariant anchor points that are common among at least some of the first set of high-elevation images; generate a mapping, based on the invariant anchor points, between pixels of the aligned high-elevation images of the first set and spatially-corresponding pixels of a second set of high-elevation images, wherein the second set of high-elevation images capture the plurality of plants at a second resolution that is greater than the first resolution; and based at least in part on the mapping, localize one or more individual plants of the plurality of plants within one or more of the second set of high-elevation images for performance of one or more agricultural tasks. ### Claim 13 (depends on 12) The system of claim 12, further including instructions to downsample the second set of high-elevation digital images to generate the first set of high-elevation digital images. ### Claim 14 (depends on 12) The system of claim 12, wherein the first set of high-elevation images are captured at a first elevation and the second set of high-elevation images are captured at a second elevation that is less than the first elevation. ### Claim 15 (depends on 12) The system of claim 12, wherein the instructions to localize include instructions to assign position coordinates to the one or more individual plants based on position coordinates generated by an airborne vehicle that acquired the first set or the second set of high-elevation images. ### Claim 16 (depends on 12) The system of claim 12, wherein the instructions to localize include instructions to map the one or more individual plants to one or more rows of a plurality of rows in which the plurality of plants are arranged. ### Claim 17 (depends on 12) The system of claim 12, further including instructions to deploy one or more agricultural robots to the one or more localized individual plants to perform one or more of the agricultural tasks. ### Claim 18 (depends on 12) The system of claim 12, wherein the first set or the second set of high-elevation images is acquired by an unmanned aerial vehicle (UAV). ### Claim 19 A non-transitory computer-readable medium for localizing one or more individual plants of a plurality of plants, the medium comprising instructions that, in response to execution of the instructions by a processor, cause the processor to: analyze a first set of high-elevation images that capture the plurality of plants at a first resolution; based on the analysis, classify a set of pixels of the first set of high-elevation images as depicting invariant anchor points, wherein the invariant anchor points include one or more visual features that are known to be movement-invariant; align high-elevation images of the first set based on one or more of the invariant anchor points that are common among at least some of the first set of high-elevation images; generate a mapping, based on the invariant anchor points, between pixels of the aligned high-elevation images of the first set and spatially-corresponding pixels of a second set of high-elevation images, wherein the second set of high-elevation images capture the plurality of plants at a second resolution that is greater than the first resolution; and based at least in part on the mapping, localize one or more individual plants of the plurality of plants within one or more of the second set of high-elevation images for performance of one or more agricultural tasks. ### Claim 20 (depends on 19) The non-transitory computer-readable medium of claim 19, further including instructions to downsample the second set of high-elevation digital images to generate the first set of high-elevation digital images.