# US 12,001,512 B2 · Generating labeled synthetic training data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,512 B2 - Document ID: US12001512B2 - Status: Granted - Kind: B2 - Application number: 17/329,528 - Filed: 2021-05-25 - Published: 2022-12-01 - Granted: 2024-06-04 - Earliest claimed: 2021-05-25 - Assignees: Mineral Earth Sciences LLC - Applicants: Mineral Earth Sciences LLC - Inventors: Kangkang Wang; Hong Wu; Bodi Yuan; Zihao Li - Web page: https://patentgrep.com/patent/US12001512B2 ## CPC - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 20/188: {Vegetation} - G06F 18/2413: based on distances to training or reference patterns - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Implementations are described herein for automatically labeling synthetic plant parts in synthetic training images, where the synthetic training images and corresponding labels can be used as training data for training machine learning models to detect, segment, and/or classify various parts of plants in digital images. In various implementations, a digital image may be obtained that captures an area. The synthetic training image may be generated to depict one or more three-dimensional synthetic plants in the area. In many implementations, a plant mask, identifying individual plants as a whole in the synthetic training image, as well as a part mask, uniquely identifying one or more parts of the synthetic plant models, can be overlaid on the synthetic training image to label the one or more parts of the synthetic plant models. ## Claims ### Claim 1 A method of generating a labeled synthetic training image, the method implemented by one or more processors and comprising: generating one or more synthetic plant models, where each synthetic plant model includes (a) a textured portion of the synthetic plant model and (b) a segmented portion of the synthetic plant model uniquely identifying individual parts of the synthetic plant; generating the synthetic training image based on (a) one or more texture portions of the one or more synthetic plant models and based on (b) a digital image that captures or synthesizes an area; and labeling the synthetic training image based on one or more of the segmented portions of the one or more synthetic plant models. ### Claim 2 (depends on 1) The method of claim 1, wherein labeling the synthetic training image based on the one or more segmented portions of the one or more synthetic plant models comprises: for each given synthetic plant model, in the one or more synthetic plant models: generating a plant mask, identifying the given synthetic plant model as a whole, based on the corresponding segmented portion of the given synthetic plant model; generating a part mask, uniquely identifying the parts of the given synthetic plant model, based on the corresponding segmented portion of the given synthetic plant model. ### Claim 3 (depends on 2) The method of claim 2, further comprising overlaying the plant and part masks on the synthetic training image to label both leaves corresponding to the given synthetic plant and the given synthetic plant as a whole. ### Claim 4 (depends on 1) The method of claim 1, wherein the segmented portion of the synthetic plan model uniquely identifies individual leaves of the synthetic plant. ### Claim 5 (depends on 4) The method of claim 4, wherein generating the textured portion of the synthetic plant model comprises: randomly selecting one or more aspects of the synthetic plant model including a number of leaves and a plant height; and for each leaf in the number of leaves, randomly selecting a leaf shape, a leaf size, and a leaf texture. ### Claim 6 (depends on 5) The method of claim 5, wherein generating the segmented portion of the synthetic plant model uniquely identifying leaves of the synthetic plant comprises: for each leaf in the number of leaves, annotating the leaf to distinguish the leaf from one or more additional leaves in the number of leaves. ### Claim 7 (depends on 6) The method of claim 6, wherein annotating the leaf to distinguish the leaf from the one or more additional leaves in the number of leaves comprises: identifying one or more pixels associated with the leaf; and annotating the one or more pixels associated with the leaf with a unique color. ### Claim 8 (depends on 1) The method of claim 1, further comprising: using the synthetic training image to train a machine learning model to segment, in digital images, plants of the same type as at least some of the synthetic plants. ### Claim 9 (depends on 1) The method of claim 1, wherein labeling the synthetic training image based on one or more segmented model portions of the one or more synthetic plants comprises: for each plant of the one or more synthetic plants, generating a bounding box for the synthetic plant; and generating an additional bounding box for each part of the synthetic plant. ### Claim 10 At least one non-transitory computer-readable medium for generating a synthetic training image, wherein the medium comprises instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to: generate one or more synthetic plant models, where each synthetic plant model includes (a) a textured portion of the synthetic plant model and (b) a segmented portion of the synthetic plant model uniquely identifying individual parts of the synthetic plant; generate the synthetic training image based on (a) one or more texture portions of the one or more synthetic plant models and based on (b) a digital image that captures or synthesizes an area; and label the synthetic training image based on one or more of the segmented portions of the one or more synthetic plant models. ### Claim 11 (depends on 10) The at least one non-transitory computer-readable medium of claim 10, wherein instructions to label the synthetic training image based on the one or more segmented portions of the one or more synthetic plant models include: for each given synthetic plant model, in the one or more synthetic plant models: generate a plant mask, identifying the given synthetic plant model as a whole, based on the corresponding segmented portion of the given synthetic plant model; generate a part mask, uniquely identifying the parts of the given synthetic plant model, based on the corresponding segmented portion of the given synthetic plant model. ### Claim 12 (depends on 11) The at least one non-transitory computer-readable medium of claim 11, further comprising instructions to overlay the plant and part masks on the synthetic training image to label both leaves corresponding to the given synthetic plant and the given synthetic plant as a whole. ### Claim 13 (depends on 10) The at least one non-transitory computer-readable medium of claim 10, wherein the segmented portion of the synthetic plan model uniquely identifies individual leaves of the synthetic plant. ### Claim 14 (depends on 13) The at least one non-transitory computer-readable medium of claim 13, wherein instructions to generate the textured portion of the synthetic plant model include: randomly select one or more aspects of the synthetic plant model including a number of leaves and a plant height; and for each leaf in the number of leaves, randomly select a leaf shape, a leaf size, and a leaf texture. ### Claim 15 (depends on 14) The at least one non-transitory computer-readable medium of claim 14, wherein instructions to generate the segmented portion of the synthetic plant model uniquely identifying leaves of the synthetic plant include: for each leaf in the number of leaves, annotate the leaf to distinguish the leaf from one or more additional leaves in the number of leaves. ### Claim 16 (depends on 15) The at least one non-transitory computer-readable medium of claim 15, wherein instructions to annotate the leaf to distinguish the leaf from the one or more additional leaves in the number of leaves include: identify one or more pixels associated with the leaf; and annotate the one or more pixels associated with the leaf with a unique color. ### Claim 17 (depends on 10) The at least one non-transitory computer-readable medium of claim 10, wherein the instructions further include: using the synthetic training image to train a machine learning model to segment, in digital images, plants of the same type as at least some of the synthetic plants. ### Claim 18 (depends on 10) The at least one non-transitory computer-readable medium of claim 10, wherein instructions to label the synthetic training image based on one or more segmented model portions of the one or more synthetic plants comprises: for each plant of the one or more synthetic plants, generate a bounding box for the synthetic plant; and generate an additional bounding box for each part of the synthetic plant. ### Claim 19 A system for generating a synthetic training image, the system comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to: generate one or more synthetic plant models, where each synthetic plant model includes (a) a textured portion of the synthetic plant model and (b) a segmented portion of the synthetic plant model uniquely identifying individual parts of the synthetic plant; generate the synthetic training image based on (a) one or more texture portions of the one or more synthetic plant models and based on (b) a digital image that captures or synthesizes an area; and label the synthetic training image based on one or more of the segmented portions of the one or more synthetic plant models. ### Claim 20 (depends on 19) The system of claim 19, wherein instructions to label the synthetic training image based on the one or more segmented portions of the one or more synthetic plant models include: for each given synthetic plant model, in the one or more synthetic plant models: generate a plant mask, identifying the given synthetic plant model as a whole, based on the corresponding segmented portion of the given synthetic plant model; generate a part mask, uniquely identifying the parts of the given synthetic plant model, based on the corresponding segmented portion of the given synthetic plant model.