# US 11,113,525 B1 · Using empirical evidence to generate synthetic training data for plant detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,113,525 B1 - Document ID: US11113525B1 - Status: Granted - Kind: B1 - Application number: 16/877,138 - Filed: 2020-05-18 - Granted: 2021-09-07 - Earliest claimed: 2020-05-18 - Assignees: X Development LLC - Applicants: X Development LLC - Inventors: Lianghao Li; Kangkang Wang; Zhiqiang Yuan - Web page: https://patentgrep.com/patent/US11113525B1 ## CPC - G06V 20/188: {Vegetation} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/24: Classification techniques - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0475: Generative networks - G06N 3/08: Learning methods - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N 3/09: Supervised learning - G06N 3/094: Adversarial learning - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06V 20/68: Food, e.g. fruit or vegetables - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Implementations are described herein for automatically generating synthetic training images that are usable as training data for training machine learning models to detect, segment, and/or classify various types of plants in digital images. In various implementations, a digital image may be obtained that captures an area. The digital image may depict the area under a lighting condition that existed in the area when a camera captured the digital image. Based at least in part on an agricultural history of the area, a plurality of three-dimensional synthetic plants may be generated. The synthetic training image may then be generated to depict the plurality of three-dimensional synthetic plants in the area. In some implementations, the generating may include graphically incorporating the plurality of three-dimensional synthetic plants with the digital image based on the lighting condition. ## Claims ### Claim 1 A method for generating a synthetic training image, the method implemented using one or more processors and comprising: obtaining a digital image that captures an area, wherein the digital image depicts the area under a lighting condition that existed in the area when a camera captured the digital image; based at least in part on an agricultural history of the area, generating a plurality of three-dimensional synthetic plants; and generating the synthetic training image to depict the plurality of three-dimensional synthetic plants in the area, wherein the generating includes graphically incorporating the plurality of three-dimensional synthetic plants with the digital image based on the lighting condition. ### Claim 2 (depends on 1) The method of claim 1, wherein the agricultural history of the area includes time-series data corresponding to a plurality of environmental conditions of the area, wherein the generating is based on the time-series data. ### Claim 3 (depends on 2) The method of claim 2, wherein the plurality of environmental conditions include two or more of temperature, precipitation, sunlight exposure, fertilizer application, soil composition, pH levels, or pesticide application. ### Claim 4 (depends on 1) The method of claim 1, wherein the agricultural history includes a plurality of environmental conditions that existed in the area during a time interval preceding capture of the digital image that captures the area. ### Claim 5 (depends on 4) The method of claim 4, wherein the graphically incorporating includes spacing the plurality of three-dimensional synthetic plants from each other in the synthetic training image based on the environmental conditions. ### Claim 6 (depends on 1) The method of claim 1, wherein the graphically incorporating includes spacing the plurality of three-dimensional synthetic plants from each other in the synthetic training image based on one or more plants already depicted in the digital image. ### Claim 7 (depends on 1) The method of claim 1, further comprising using the synthetic training image to train a machine learning model to segment or detect, in digital images, plants of a same type as at least some of the three-dimensional synthetic plants. ### Claim 8 At least one non-transitory computer-readable medium for generating a synthetic training image, wherein the medium comprises instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to: obtain a digital image that captures an area, wherein the digital image depicts the area under a lighting condition that existed in the area when a camera captured the digital image; based at least in part on an agricultural history of the area, generate a plurality of three-dimensional synthetic plants; and generate the synthetic training image to depict the plurality of three-dimensional synthetic plants in the area, wherein the generation includes graphical incorporation of the plurality of three-dimensional synthetic plants with the digital image based on the lighting condition. ### Claim 9 (depends on 8) The at least one non-transitory computer-readable medium of claim 8, wherein the agricultural history of the area includes time-series data corresponding to a plurality of environmental conditions of the area, wherein the generating is based on the time-series data. ### Claim 10 (depends on 9) The at least one non-transitory computer-readable medium of claim 9, wherein the plurality of environmental conditions include two or more of temperature, precipitation, sunlight exposure, fertilizer application, soil composition, pH levels, or pesticide application. ### Claim 11 (depends on 8) The at least one non-transitory computer-readable medium of claim 8, wherein the agricultural history includes a plurality of environmental conditions that existed in the area during a time interval preceding capture of the digital image that captures the area. ### Claim 12 (depends on 11) The at least one non-transitory computer-readable medium of claim 11, wherein the graphical incorporation includes spacing of the plurality of three-dimensional synthetic plants from each other in the synthetic training image based on the environmental conditions. ### Claim 13 (depends on 8) The at least one non-transitory computer-readable medium of claim 8, wherein the graphical incorporation includes spacing of the plurality of three-dimensional synthetic plants from each other in the synthetic training image based on one or more plants already depicted in the digital image. ### Claim 14 (depends on 8) The at least one non-transitory computer-readable medium of claim 8, further comprising instructions to use the synthetic training image to train a machine learning model to segment or detect, in digital images, plants of a same type as at least some of the three-dimensional synthetic plants. ### Claim 15 A system for generating a synthetic training image, the system comprising one or more processors and memory storing instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to: obtain a digital image that captures an area, wherein the digital image depicts the area under a lighting condition that existed in the area when a camera captured the digital image; based at least in part on an agricultural history of the area, generate a plurality of three-dimensional synthetic plants; and generate the synthetic training image to depict the plurality of three-dimensional synthetic plants in the area, wherein the generation includes graphical incorporation of the plurality of three-dimensional synthetic plants with the digital image based on the lighting condition. ### Claim 16 (depends on 15) The system of claim 15, wherein the agricultural history of the area includes time-series data corresponding to a plurality of environmental conditions of the area, wherein the generating is based on the time-series data. ### Claim 17 (depends on 16) The system of claim 16, wherein the plurality of environmental conditions include two or more of temperature, precipitation, sunlight exposure, fertilizer application, soil composition, pH levels, or pesticide application. ### Claim 18 (depends on 15) The system of claim 15, wherein the agricultural history includes a plurality of environmental conditions that existed in the area during a time interval preceding capture of the digital image that captures the area. ### Claim 19 (depends on 18) The system of claim 18, wherein the graphical incorporation includes spacing of the plurality of three-dimensional synthetic plants from each other in the synthetic training image based on the environmental conditions. ### Claim 20 (depends on 15) The system of claim 15, wherein the graphical incorporation includes spacing of the plurality of three-dimensional synthetic plants from each other in the synthetic training image based on one or more plants already depicted in the digital image.