# US 10,769,845 B2 · System and method of virtual plant field modelling > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,845 B2 - Document ID: US10769845B2 - Status: Granted - Kind: B2 - Application number: 15/996,810 - Filed: 2018-06-04 - Published: 2018-10-04 - Granted: 2020-09-08 - Earliest claimed: 2015-05-18 - Assignees: Blue River Technology Inc. - Applicants: Blue River Technology Inc. - Inventors: Lee Kamp Redden; Nicholas Apostoloff - Web page: https://patentgrep.com/patent/US10769845B2 ## CPC - G06T 17/05: Geographic models - G06T 7/11: Region-based segmentation - G06T 7/33: using feature-based methods - G06T 7/337: {involving reference images or patches} - G06T 7/344: {involving models} - G06T 7/74: {involving reference images or patches} - G06T 15/08: Volume rendering - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G06T 7/194: involving foreground-background segmentation - G06T 2210/56: Particle system, point based geometry or rendering - G06T 2219/2008: Assembling, disassembling - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A technique for generating virtual models of plants in a field is described. Generally, this includes recording images of plants in-situ; generating point clouds from the images; generating skeleton segments from the point cloud; classifying a subset of skeleton segments as unique plant features using the images; and growing plant skeletons from skeleton segments classified as unique plant feature. The technique may be used to generate a virtual model of a single, real plant, a portion of a real plant field, and/or the entirety of the real plant field. The virtual model can be analyzed to determine or estimate a variety of individual plant or plant population parameters, which in turn can be used to identify potential treatments or thinning practices, or predict future values for yield, plant uniformity, or any other parameter can be determined from the projected results based on the virtual model. ## Claims ### Claim 1 A method comprising: accessing an image of a plant recorded by an imaging system passing by the plant; generating a point cloud of the plant comprising a set of virtual points reduced from the captured image; generating a plurality of skeleton segments based on the virtual points of the point cloud, each skeleton segment representing an individual plant segment of the plant and comprising one or more nodes; identifying a set of candidate skeleton segments of the plurality of skeleton segments within a section of the image corresponding to a unique plant feature of the plant, each identified candidate skeleton segment located near another section of the image corresponding to a ground plane; building a plant virtual model by connecting candidate skeleton segments within the section of the image corresponding to the plant feature of the plant; determining a treatment for the unique plant feature using the plant virtual model; and generating instructions to apply the treatment to the unique plant feature of the plant. ### Claim 2 (depends on 1) The method of claim 1, further comprising: building a plant virtual model for each plant of a plurality of plants in a geographic region, wherein the plurality of plants comprises the plant. ### Claim 3 (depends on 2) The method of claim 2, further comprising: generating a virtual model for the geographic region based on the plant virtual models for the plurality of plants. ### Claim 4 (depends on 3) The method of claim 3, wherein the virtual model for the geographic region is generated by mosaicking the plant virtual models for the plurality of plants. ### Claim 5 (depends on 2) The method of claim 2, further comprising: generating aggregate parameters for the plurality of plants based on the virtual plant models for the plurality of plants. ### Claim 6 (depends on 5) The method of claim 5, wherein the aggregate parameters comprise at least one of: plant density, a number of plants within the geographic region, plant size, plant greenness, plant uniformity, or terminal yield. ### Claim 7 (depends on 5) The method of claim 5, further comprising: determining an action to take with respect to the plurality of plants based on the aggregate parameters for the plurality of plants. ### Claim 8 (depends on 7) The method of claim 7, wherein the action comprises removing one or more plants of the plurality of plants. ### Claim 9 (depends on 7) The method of claim 7, wherein the action comprises harvesting one or more plants of the plurality of plants. ### Claim 10 A plant treatment system comprising: an imaging system comprising at least one camera configured to capture an image of a plant as the plant treatment system passes by the plant; a non-transitory computer readable storage medium storing instructions that, when executed by a processor, cause the processor to: generate a point cloud of the plant comprising a set of virtual points by reducing the image; generate a plurality of skeleton segments based on the point cloud, each skeleton segment representing an individual plant segment of the plant and comprising one or more nodes; build a plant virtual model by connecting candidate skeleton segments within the section of the image corresponding to the plant feature of the plant; determine a treatment for the unique plant feature using the plant virtual model; and generate instructions to apply the treatment to the unique plant feature of the plant. ### Claim 11 (depends on 10) The system of claim 10, further comprising instructions to: build a plant virtual model for each plant of a plurality of plants in a geographic region, wherein the plurality of plants comprises the plant. ### Claim 12 (depends on 11) The system of claim 11, further comprising instructions to: generate a virtual model for the geographic region based on the plant virtual models for the plurality of plants. ### Claim 13 (depends on 12) The system of claim 12, wherein the virtual model for the geographic region is generated by mosaicking the plant virtual models for the plurality of plants. ### Claim 14 (depends on 11) The system of claim 11, further comprising instructions to: generate aggregate parameters for the plurality of plants based on the virtual plant models for the plurality of plants, wherein the aggregate parameters comprise at least one of: plant density, a number of plants within the geographic region, plant size, plant greenness, plant uniformity, or terminal yield. ### Claim 15 A non-transitory computer readable storage medium storing instructions that, when executed by a processor, cause the processor to: capture an image of a plant using an imaging system passing by the plant; generate a point cloud of the plant based on the image, wherein the point cloud reduces the image of the plant to a set of virtual points and comprising one or more nodes; generate a plurality of skeleton segments based on the point cloud, each skeleton segment representing an individual plant segment of the plant; identify a set of candidate skeleton segments of the plurality of skeleton segments within a section of the image corresponding to a unique plant feature of the plant, each identified candidate skeleton segment located near another section of the image corresponding to a ground plane; build a plant virtual model by connecting candidate skeleton segments within the section of the image corresponding to the plant feature of the plant; determine a treatment for the unique plant feature using the plant virtual model; and apply the treatment to the unique plant feature of the plant. ### Claim 16 (depends on 15) The non-transitory computer readable medium of claim 15, further comprising instructions to: build a plant virtual model for each plant of a plurality of plants in a geographic region, wherein the plurality of plants comprises the plant. ### Claim 17 (depends on 16) The non-transitory computer readable medium of claim 16, further comprising instructions to: generate a virtual model for the geographic region based on the plant virtual models for the plurality of plants. ### Claim 18 (depends on 17) The non-transitory computer readable medium of claim 17, wherein the virtual model for the geographic region is generated by mosaicking the plant virtual models for the plurality of plants. ### Claim 19 (depends on 15) The non-transitory computer readable medium of claim 15, further comprising instructions to: generate aggregate parameters for the plurality of plants based on the virtual plant models for the plurality of plants.