# US 12,757,812 B2 · Image segmentation for row following and associated training system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,757,812 B2 - Document ID: US12757812B2 - Status: Granted - Kind: B2 - Application number: 18/869,592 - Filed: 2023-05-25 - Published: 2025-09-25 - Granted: 2026-10-13 - Earliest claimed: 2022-05-26 - Assignees: Bonsai Robotics Inc. - Applicants: Bonsai Robotics Inc. - Inventors: Ethan Rublee; Gary Bradski; Wesley Chaney; E. Riba; Kyle Coble; Hauke Strasdat; Wren Ramsey - Web page: https://patentgrep.com/patent/US12757812B2 ## CPC - A01B 69/008: {automatic} - A01B 69/001: {Steering by means of optical assistance, e.g. television cameras (steering devices for road marking vehicles E01C23/163)} - G05D 1/243: Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals (using passive navigation aids external to the vehicle G05D1/244; using signals from positioning sensors located off-board the vehicle G05D1/249) - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06V 20/70: Labelling scene content, e.g. deriving syntactic or semantic representations - A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Methods and systems related to computer vision for agricultural applications are disclosed herein. A disclosed method for navigating a robot along a crop row, in which each step is computer-implemented by a navigation system for the robot, includes capturing an image of at least a portion of the crop row, labeling, using a segmentation network, a portion of the image with a label, deriving a navigation path from the portion of the image and the label, generating a control signal for the autonomous navigation system to follow the navigation path, and navigating the robot along the crop row using the control signal. ## Claims ### Claim 1 A method for navigating a robot along a crop row, in which each step is computer-implemented by a navigation system for the robot, the method comprising: training a segmentation network using a set of images of a set of crop rows and a set of synthesized training images generated using the set of images, wherein the set of images comprises 100 or fewer images and wherein the set of synthesized training images comprises 500 or more images; capturing an image of at least a portion of a crop row of the set of crop rows; labeling, using the segmentation network, a portion of the image with a label; deriving a navigation path from the portion of the image and the label; generating a control signal for the navigation system to follow the navigation path; and navigating the robot along the crop row using the control signal. ### Claim 2 (depends on 1) The method of claim 1, further comprising: checking the label for the portion of the image using an expected geometric principle associated with navigating sets of crop rows; and overriding the label if the expected geometric principle is violated. ### Claim 3 (depends on 2) The method of claim 2, further comprising: capturing depth information from the crop row; wherein the checking of the label using the expected geometric principle includes using the depth information. ### Claim 4 (depends on 2) The method of claim 2, wherein: the label is for an inter-crop row path; and the expected geometric principle associated with navigating sets of crop rows regards a pair of edges of the portion of the image. ### Claim 5 (depends on 1) The method of claim 1, wherein: the generating of the control signal is conducted using a proportional-integral-derivative controller; and the navigation path provides a set point for the proportional-integral-derivative controller. ### Claim 6 (depends on 1) The method of claim 1, wherein generating the control signal includes: translating the navigation path into a frame of reference; capturing a second image of the crop row, wherein the second image is registered in the frame of reference; and projecting the navigation path onto the second image. ### Claim 7 (depends on 1) The method of claim 1, further comprising, prior to capturing the image: displaying the set of images on a user interface; accepting a set of label inputs on the set of images on the user interface; and training the segmentation network using the set of label inputs and the set of images. ### Claim 8 (depends on 1) The method of claim 1, wherein the set of crop rows are one of: (i) on a single farm; or (ii) share a single crop type. ### Claim 9 (depends on 1) The method of claim 1, wherein generating the set of synthesized training images includes conducting at least one operation on the set of images selected from: warping, blurring, relighting, and recoloring. ### Claim 10 (depends on 1) The method of claim 1, further comprising: labeling, using the segmentation network, a second portion of the image with a second label; wherein: (i) the label is for an inter-crop row path; (ii) the second label is for a potential obstruction; and (iii) navigating the robot along the crop row using the control signal includes at least temporarily stopping the robot to avoid a collision. ### Claim 11 (depends on 1) The method of claim 1, wherein: the image is captured by at least one imager on the robot; and the segmentation network is computer-implemented on the robot. ### Claim 12 (depends on 1) The method of claim 1, wherein: the robot includes at least two imagers; the image is captured using the at least two imagers; and the image includes depth information. ### Claim 13 A navigation system for navigating a robot along a crop row comprising: a sensor; an actuator on the robot; a means for capturing a set of images of a set of crop rows including an image of at least a portion of the crop row using the sensor; a means for generating, using the set of images, a larger set of synthesized training images; a means for training a segmentation network for labeling a portion of the image with a label using the set of images and the set of synthesized training images, wherein the set of images comprises 100 or fewer images and wherein the set of synthesized training images comprises 500 or more images; a means for deriving a navigation path from the portion of the image with the label; and a means for generating a control signal, for the actuator, to cause the robot to follow the navigation path. ### Claim 14 (depends on 13) The navigation system of claim 13, further comprising: a means for checking the label for the portion of the image using an expected geometric principle associated with navigating sets of crop rows; and a means for overriding the label if the expected geometric principle is violated. ### Claim 15 (depends on 14) The navigation system of claim 14, further comprising: a means for capturing depth information from the crop row; wherein the checking of the label using the expected geometric principle includes using the depth information. ### Claim 16 (depends on 13) The navigation system of claim 13, further comprising: a means for translating the navigation path into a frame of reference; a means for capturing a second image of the crop row, wherein the second image is registered in the frame of reference; and a means for projecting the navigation path onto the second image. ### Claim 17 (depends on 13) The navigation system of claim 13, wherein the set of crop rows are one of: (i) on a single farm; or (ii) share a single crop type.