Semantic segmentation to identify and treat plants in a field and verify the plant treatments

A non-transitory computer readable storage medium comprising instructions for treating a plant in a field as a farming machine travels through the field, the farming machine including an imaging system to capture images of plants in the field as the farming machine travels through the field, the instructions, when executed by a processor, causing the processor to:

• access an image captured by the imaging system, the accessed image having a pixel dimensionality and including pixels representing the plant in the field;
• apply a semantic segmentation model to the image to identify which pixels of the image represent the plant, the semantic segmentation model including a plurality layers in a convolutional neural network, and the semantic segmentation model:
• encoding the image onto a first layer as an encoded image, the first layer having the pixel dimensionality;
• reducing the encoded image to a reduced image, the reduced image on a second layer and having an identification dimensionality;
• identifying latent features in the reduced image as the plant;
• decoding the reduced image to a treatment map, the treatment map on a third layer having a dimensionality substantially similar to a dimensionality of a plurality of plant treatment mechanisms coupled to the farming machine, and
• identifying elements of the treatment map including the plant, each element corresponding to a plant treatment mechanism;
• generate a set of plant treatment instructions for the plurality of plant treatment mechanisms based on identified elements of the treatment map including the plant;
• send the plant treatment instructions to the plurality of plant treatment mechanisms;
• and
• actuate the plant treatment mechanisms using the plant treatment instructions such that the identified plant is treated with the plant treatment mechanisms.

A farming machine comprising:

• a plurality of plant treatment mechanisms for treating a plant as the farming machine travels past the plant in a field;
• an image acquisition system to capture an image of the plant in the field as the farming machine moves through the field, the image having a pixel dimensionality and comprising pixels representing the plant in the field;
• and
• a control system including for applying, using a processor, a semantic segmentation model to the image to identify which pixels of the image represent the plant, the semantic segmentation model including a plurality layers in a convolutional neural network, and the semantic segmentation model:
• encoding the image onto a first layer as an encoded image, the first layer having the pixel dimensionality;
• reducing the encoded image to a reduced image, the reduced image on a second layer and having an identification dimensionality;
• identifying latent features in the reduced image as the plant;
• decoding the reduced image to a treatment map, the treatment map on a third layer having a dimensionality substantially similar to a dimensionality of the plurality of plant treatment mechanisms,
• identifying elements of the treatment map including the plant, each element corresponding to a plant treatment mechanism,
• generating a set of plant treatment instructions for the plurality of plant treatment mechanisms based on identified elements of the treatment map including the plant,
• sending the plant treatment instructions to the plurality of plant treatment mechanisms, and
• actuating the plant treatment mechanisms using the plant treatment instructions such that the identified plant is treated with the plant treatment mechanisms.