# US 2022/0030759 A1 · Method, system and robot for autonomous navigation thereof between two rows of plants > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0030759 A1 - Document ID: US20220030759A1 - Status: Published application - Kind: A1 - Application number: 17/388,169 - Filed: 2021-07-29 - Published: 2022-02-03 - Earliest claimed: 2020-07-31 - Assignees: Universitat Politècnica de València - Applicants: Universitat Politecnica De Valencia - Inventors: Francisco ROVIRA MAS; Andrés CUENCA CUENCA; Verónica SAIZ RUBIO - Web page: https://patentgrep.com/patent/US20220030759A1 ## CPC - A01B 69/008: {automatic} - G05D 1/0255: {using acoustic signals, e.g. ultra-sonic singals (sonar systems designed for anti-collision purposes G01S15/93)} - B60W 50/0098: {Details of control systems ensuring comfort, safety or stability not otherwise provided for} - B60W 60/0025: {specially adapted for specific operations} - G05D 1/0257: {using a radar (radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931)} - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - B60W 2050/0031: {Mathematical model of the vehicle} - B60W 2300/15: Agricultural vehicles - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2420/50: Magnetic or electromagnetic sensors - B60W 2420/54: Audio sensitive means, e.g. ultrasound - A01B: SOIL WORKING IN AGRICULTURE OR FORESTRY; PARTS, DETAILS, OR ACCESSORIES OF AGRICULTURAL MACHINES OR IMPLEMENTS, IN GENERAL (making or covering furrows or holes for sowing, planting or manuring A01C5/00; mowers convertible to soil working apparatus or capable of soil working A01D42/04; mowers combined with soil working implements A01D43/12; soil working for engineering purposes E01, E02, E21) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method, system and robot for autonomous navigation thereof between two rows of plants, wherein said robot includes two or more sensing devices, sensor A and sensor B, mounted thereon and moves forward along an axis parallel to the rows of plants, being autonomously steered by exerting angular corrections to place the robot as close as possible to the centerline between the rows of plants, wherein the method and system includes the following: (ii) dividing a two-dimensional grid of square cells into IG·JG groups of cells; (iii) obtaining data points using sensor A and sensor B; (vii) moving the robot: (a) by turning right; or (b) by turning left; or (c) forward without turning, depending on whether each group of cells (i,j) is calculated as low-activated, high-activated or not activated using said data points. ## Claims ### Claim 1 A method for autonomous navigation of a robot between two rows of plants spaced Q metres apart, wherein said robot comprises two sensing devices, sensor A and sensor B, mounted at a position O thereon and moves forward along a y-axis being autonomously steered by exerting angular corrections to place the robot as close as possible to the centerline, wherein each sensing device is a device which detects electromagnetic waves or detects sound waves, and wherein when both devices detect: (A) electromagnetic waves, sensor A detects said waves at a frequency and/or field-of-view different from that of the waves detected by sensor B; and (B) sound waves, sensor A detects said waves at a frequency and/or field-of-view different from that of the waves detected by sensor B, and wherein said y-axis is a horizontal axis and said x-axis is a horizontal axis perpendicular to said y-axis, and wherein said method comprises the following steps: (i) defining a two-dimensional grid of square cells in said plane, wherein said grid is XG cells in width and YG cells in length, and said cells have sides of length c, wherein: (a) said width extends horizontally |Xmin| metres away from O to the left along said x-axis and horizontally Xmax metres away from O to the right along said x-axis; and (b) said length extends horizontally Ymax metres away from O along said y-axis; wherein: Xmin=−XG·c/2; Xmax=XG·c/2; Ymax=YG·c; XG is a whole number selected from between 2 and 1000; YG is a whole number selected from between 3 and 1500; and c is a number selected from between 0.01 and 0.5 m; and (c) each cell (h,v) is assigned a coordinate comprising a first number, h, and a second number, v, wherein h is a number from 1 to XG and v is a number from 1 to YG, wherein: h=h′+1; and v=v′+1; wherein: h′ is the number of said cells, counted along said x-axis starting from the left-most cell, which separate said cell (h,v) from the outside of said two-dimensional grid; and v′ is the number of said cells, counted along said y-axis starting from the cell that is most remote from said robot, which separate said cell (h,v) from the outside of said two-dimensional grid; (ii) dividing the two-dimensional grid of square cells into IG·JG groups of cells, wherein each group is XG/JG cells in width and YG/IG cells in length, wherein: (a) IG is 3; (b) JG is 2; and (c) each group of cells (i,j) (omij) is assigned a coordinate comprising a first number, i, and a second number, j, wherein i is a number from 1 to IG and j is a number from 1 to JG, wherein i=i′+1, and j=j′+1, wherein: i′ is the number of said groups of cells, counted along said y-axis starting from the group of cells that is most remote from said robot, which separate said group of cells from the outside of said two-dimensional grid; and j′ is the number of said groups of cells, counted along said x-axis starting from the left-most group of cells, which separate said group of cells (i,j) (omij) from the outside of said two-dimensional grid; (iii) obtaining data points using sensor A and data points using sensor B, wherein each data point is the point in space at which electromagnetic or sound waves are reflected from a surface located within the rectangular cuboid volume bounded by the dimensions of said two-dimensional grid in the x-axis and y-axis and having a height of Zmax−Zmin metres along a z-axis, wherein said z-axis is a vertical axis perpendicular to said x-axis and said y-axis, and said surface is the surface of an object, wherein: Zmin is a value selected from −0.5 to 0.5 metres; Zmax is a value selected from 1 to 5 metres; wherein: (a) each data point obtained using sensor A is assigned a coordinate comprising a first number, xL, and a second number, yL wherein: xL is the distance along the x-axis from said data point to O; and yL is the distance along the y-axis from said data point to O, wherein the coordinate of position O is (0, 0); and (b) each data point obtained using sensor B is assigned a coordinate comprising a first number, xV, and a second number, yV wherein: xV is the distance along the x-axis from said data point to O; and yV is the distance along the y-axis from said data point to O, wherein the coordinate of position O is (0, 0); (iv) converting each data point into a discretized data point, wherein: (a) data point (xL, yL) is converted to a discretized data point (LV, LH), according to the following formulae: LV=∥yL/c∥; and LH=∥(xL−Xmin)/c∥; (b) data point (xV, yV) is converted to a discretized data point (VV, VH) according to the following formulae: VV=∥yV/c∥; and VH=∥(xV−Xmin)/c∥; (v) calculating a fusion function value, Γ(h,v), for each cell (h,v) of said two-dimensional grid according to the following formula (I): Γ(h,v)=KV·γV(VV,VH)+KL·γL(LV,LH) (I) wherein: KL is a non-negative integer selected from between 0 and 5; γL(LV, LH) is a whole number calculated from the number DN(LH, LV) of discretized data points (LV, LH) for which LV=v and LH=h; KV is a non-negative integer selected from between 0 and 5; γV(VV, VH) is a whole number calculated from the number DN(VH, VV) of discretized data points (VV, VH) for which VV=v and VH=h; (vi) calculating for each group of cells (i,j) (omij): (a) the cumulative total of fusion function values, CUMij(h), of all cells in said group of cells which have a coordinate with the same h value; and (b) the sum of all cumulative totals of fusion function values, SUMij, in said group of cells; wherein when: SUMij≥∥0.4·OM∥ said group of cells is classified as high-activated; and SUMij≥∥0.2·OM∥ and <∥0.4·OM∥ said sub-volume is classified as low-activated, wherein OM is the maximum SUM value determined for any group of cells (i,j) (omij) in said two-dimensional grid; wherein (vii) the robot moves: (a) through a plane defined by said y-axis and said x-axis towards a target point Pt placed ahead of the robot, on the centerline parallel to the detected left row and separated from it by a distance Q/2: when out of the set SR consisting of the groups of cells (1, 1), (2, 1), (3, 1) and (1, 2): om11 is high-activated, om11 is high-activated and om21 is low-activated, om11 is low-activated and om21 is high-activated, om21 is low-activated, om21 is low-activated and om31 is low-activated, om31 is low-activated, or om12 is low-activated and om21 is high-activated, and no other group of cells in said set SR or the groups of cells (2, 2) or (3, 2) is activated; or (b) through a plane defined by said y-axis and said x-axis towards a target point Pt placed ahead of the robot, on the centerline parallel to the detected right row and separated from it by a distance Q/2: when out of the set SL consisting of the groups of cells (1, 2), (2, 2), (3, 2) and (1, 1): om12 is high-activated, om12 is high-activated and om22 is low-activated, om12 is low-activated and om22 is high-activated, om22 is low-activated, om22 is low-activated and om32 is low-activated, om32 is low-activated, or om11 is low-activated and om22 is high-activated, and no other group of cells in said set SL or the groups of cells (2, 1) or (3, 1) is activated; or (c) through a plane defined by said y-axis and said x-axis towards a target point Pt placed ahead of the robot, on the centerline equidistant to both detected rows: when out of the set SR consisting of the groups of cells (1, 1), (2, 1), (3, 1) and (1, 2): om11 is high-activated, om11 is high-activated and om21 is low-activated, om11 is low-activated and om21 is high-activated, om21 is low-activated, om21 is low-activated and group of cells (3, 1) is low-activated, om31 is low-activated, or om12 is low-activated and om21 is high-activated, and no other group of cells in said set SR or the groups of cells (2, 2) or (3, 2) is activated; and when out of the set SL consisting of the groups of cells (1, 2), (2, 2), (3, 2) and (1, 1): om12 is high-activated, om12 is high-activated and om22 is low-activated, om12 is low-activated and om22 is high-activated om22 is low-activated, om22 is low-activated and om32 is low-activated, om32 is low-activated, omit is low-activated, om22 is high-activated and no other group of cells in said set SL or the groups of cells (2, 1) or (3, 1) is activated. ### Claim 2 (depends on 1) The method according to claim 1, wherein step (vi) also comprises calculating for each group of cells (i,j) (omij) for which SUMij≥1, an expected row position value, Lij, according to the following formula (II): Lij=∥XG/JG−Mij/SUMij∥ (II) wherein Mij, is the sum of t values of all cells in said group of cells which have a coordinate with the same h value, wherein said t values are calculated according to the following formula (III): t=(∥XG/JG∥−h)·CUMij(h) (III) wherein: (a) when the robot moves through a plane defined by said y-axis and said x-axis by turning towards the target point Pt which is Q/2 metres to the right of row boundary RBL at a look-ahead distance of Ymax·r metres from O, wherein: RBL is the weighted average Lij of the groups of cells (i,j) which have a coordinate with the j value of 1; and r is between 0.5 and 0.9; (b) when the robot moves through a plane defined by said y-axis and said x-axis by turning towards the target point Pt which is Q/2 metres to the left of row boundary RBR at a look-ahead distance of Ymax·r metres from O, wherein: RBR is the weighted average Lij of the groups of cells (i,j) which have a coordinate with the j value of 2; (c) when the robot moves through a plane defined by said y-axis and said x-axis by turning towards the target point Pt which is placed equidistant between RBL and RBR. ### Claim 3 (depends on 1) The method according to claim 1, wherein sensor A is a multi-beam lidar and sensor B is a stereoscopic or time-of-flight 3D camera. ### Claim 4 (depends on 3) The method according claim 3, wherein: (a) γL(LV, LH) is the whole number DN(LH, LV) of discretized data points (LV, LH) for which LV=v and LH=h; and (b) γV(VV, VH) is: 1 when DN(VH, VV)>THV·DN(VH, VV)max; or 0 when DN(VH, VV) THV·DN(VH, VV)max wherein DN(VH, VV) is: the whole number of discretized data points (VV, VH) when VV·c≤3; or a whole number calculated from the number of discretized data points (VV, VH) for which VV=v, VH=h and VV·c>3, multiplied by (VV·c/3)2; and wherein: THV is a threshold value, wherein 0≤THV≤1; and DN(VH, VV)max is the maximum value of DN(VH, VV) determined for any cell (h,v) in said two-dimensional grid. ### Claim 5 (depends on 1) The method according to claim 1, wherein the robot additionally comprises a sonar sensor mounted at a position O thereon for safeguarding against collisions, which obtains a front range and/or a rear range wherein: (a) each front range is the point in space at which sound is reflected from a surface located to the front of said robot within a spherical sector of radius |Xmax| metres having a cone angle of less than 45° centred along the y-axis, and wherein FS′ is the distance along the y-axis from said range to O; and (b) each rear range is the point in space at which sound is reflected from a surface located to the back of said robot within a spherical sector of radius |Xmax| metres having a cone angle of less than 45° centred along the y-axis, and wherein BS′ is the distance along the y-axis from said range to O, and said method additionally comprises determining FS and BS, wherein FS is the minimum FS′, and BS is the minimum BS′, and the robot stops moving forward when FS is THV·DN(VH, VV)max; or 0 when DN(VH, VV) THV·DN(VH, VV)max wherein DN(VH, VV) is: the whole number of discretized data points (VV, VH) when VV·c≤3; or a whole number calculated from the number of discretized data points (VV, VH) for which VV=v, VH=h and VV·c>3, multiplied by (VV·c/3)2; and wherein: THV is a threshold value, wherein 0≤THV≤1; and DN(VH, VV)max is the maximum value of DN(VH, VV) determined for any cell (h,v) in said two-dimensional grid. ### Claim 11 (depends on 7) The system according to claim 7, wherein the robot additionally comprises a sonar sensor mounted at a position O thereon for safeguarding against collisions, which obtains a front range and/or a rear range wherein: (a) each front range is the point in space at which sound is reflected from a surface located to the front of said robot within a spherical sector of radius |Xmax| metres having a cone angle of less than 45° centred along the y-axis, and wherein FS′ is the distance along the y-axis from said range to O; and (b) each rear range is the point in space at which sound is reflected from a surface located to the back of said robot within a spherical sector of radius |Xmax| metres having a cone angle of less than 45° centred along the y-axis, and wherein BS′ is the distance along the y-axis from said range to O, and said method additionally comprises determining FS and BS, wherein FS is the minimum FS′, and BS is the minimum BS′, and the robot stops moving forward when FS is