# US 2023/0410489 A1 · Method and system for sensor fusion in vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0410489 A1 - Document ID: US20230410489A1 - Status: Published application - Kind: A1 - Application number: 17/954,835 - Filed: 2022-09-28 - Published: 2023-12-21 - Earliest claimed: 2022-06-17 - Assignees: Hyundai Motor Company; Kia Corporation - Applicants: Hyundai Motor Company; Kia Corporation - Inventors: Bo Young Yun; Nam Hyung Lee - Web page: https://patentgrep.com/patent/US20230410489A1 ## CPC - G06V 10/803: {of input or preprocessed data} - G01S 13/726: {Multiple target tracking} - G01S 13/865: {Combination of radar systems with lidar systems} - G01S 13/867: {Combination of radar systems with cameras} - G01S 17/66: Tracking systems using electromagnetic waves other than radio waves - G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders - G06V 10/811: {the classifiers operating on different input data, e.g. multi-modal recognition} - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G01S 13/931: of land vehicles - G01S 17/931: of land vehicles - G01S 2013/9323: {Alternative operation using light waves} - G01S 2013/93271: {in the front of the vehicles} - G01S 2013/93274: {on the side of the vehicles} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES ## Abstract A method for sensor fusion in a vehicle comprises generating a track for a target vehicle based on data acquired through a surround view monitoring camera of the vehicle, acquiring a plurality of pieces of first point data for an object around the vehicle, identifying a plurality of pieces of second point data associated with the track from among the plurality of pieces of first point data, and updating a sensor fusion track generated for the target vehicle using the plurality of pieces of second point data. ## Claims ### Claim 1 A method for sensor fusion in a vehicle, the method comprising: generating a track for a target vehicle based on data acquired through a surround view monitoring camera of a vehicle; acquiring a plurality of pieces of first point data for an object around the vehicle; identifying a plurality of pieces of second point data associated with the track from among the plurality of pieces of first point data; and updating a sensor fusion track generated for the target vehicle using the plurality of pieces of second point data. ### Claim 2 (depends on 1) The method according to claim 1, wherein the sensor fusion track is pre-generated for the target vehicle based on sensor data acquired through a sensing device of the vehicle. ### Claim 3 (depends on 1) The method according to claim 1, wherein the target vehicle is located on a side or rear side of the vehicle. ### Claim 4 (depends on 3) The method according to claim 3, wherein the identifying comprises: calculating a distance value from a center point of the track to any one corner of the track based on a width and a length of the track; and identifying the plurality of pieces of second point data associated with the track from among the plurality of pieces of first point data based on the distance value. ### Claim 5 (depends on 4) The method according to claim 4, wherein the identifying comprises: determining the distance value as a radius; and identifying the plurality of pieces of second point data located within the radius from the center point of the track from among the plurality of pieces of first point data. ### Claim 6 (depends on 3) The method according to claim 3, further comprising: selecting point data at a first location and point data at a second location among the plurality of pieces of second point data based on an angle formed by a straight line connecting each of the plurality of pieces of second point data from an origin and an X-axis in a vehicle coordinate system having a designated location in the vehicle as the origin, wherein the updating comprises updating location information of the sensor fusion track based on information about the first location and the second location. ### Claim 7 (depends on 6) The method according to claim 6, wherein: the point data at the first location comprises point data having a smallest value as the angle among the plurality of pieces of second point data, and the point data at the second location comprises point data having a largest value as the angle among the plurality of pieces of second point data. ### Claim 8 (depends on 6) The method according to claim 6, wherein the updating comprises: generating a two-dimensional equation representing a straight line connecting the first location and the second location; identifying a first coordinate value of a first corner among corners of the sensor fusion track; determining a second coordinate value of a corner corresponding to the first corner among corners of the target vehicle based on the two-dimensional equation and the first coordinate value; and calculating a third coordinate value corresponding to a location of a midpoint of a rear bumper of the target vehicle, thereby updating location information of the sensor fusion track. ### Claim 9 (depends on 8) The method according to claim 8, wherein the first corner comprises: a corner located on a lower right side of a center of the sensor fusion track; a corner located on a lower left side of the center of the sensor fusion track; or a corner at a shortest distance from a straight line connecting the first location and the second location among the corners of the sensor fusion track. ### Claim 10 (depends on 8) The method according to claim 8, wherein the updating comprises: identifying the second coordinate value having the same Y-axis coordinate value as a Y-axis coordinate value of the first coordinate value in the straight line connecting the first location and the second location; calculating a heading angle of the sensor fusion track to be updated based on a slope of the straight line connecting the first location and the second location; and calculating the third coordinate value based on a half value of a width of the track and the calculated heading angle. ### Claim 11 (depends on 10) The method according to claim 10, wherein: the heading angle is calculated by applying a slope value of the two-dimensional equation as a variable of an arc tangent function, an X-axis value of the third coordinate value is calculated based on a value calculated by applying the heading angle as a variable of a sine function, an X-axis value of the second coordinate value, and the half value of the width of the track, and a Y-axis value of the third coordinate value is calculated based on a value calculated by applying the heading angle as a variable of a cosine function, a Y-axis value of the second coordinate value, and the half value of the width of the track. ### Claim 12 A system for sensor fusion in a vehicle, the system comprising: an interface configured to receive data from a sensing device of a vehicle; a memory configured to store data of a sensor fusion track generated for a target vehicle; and a processor electrically connected to the interface and the memory, wherein the processor is configured to: generate a track for the target vehicle based on data of a surround view monitoring camera of the vehicle received through the interface; acquire a plurality of pieces of first point data for an object around the vehicle; identify a plurality of pieces of second point data associated with the track from among the plurality of pieces of first point data; and update the sensor fusion track using the plurality of pieces of second point data. ### Claim 13 (depends on 12) The system according to claim 12, wherein the target vehicle is located on a side or rear side of the vehicle. ### Claim 14 (depends on 13) The system according to claim 13, wherein the processor is configured to: calculate a distance value from a center point of the track to any one corner of the track based on a width and a length of the track; and identify the plurality of pieces of second point data associated with the track from among the plurality of pieces of first point data based on the distance value. ### Claim 15 (depends on 14) The system according to claim 14, wherein the processor is configured to: determine the distance value as a radius; and identify the plurality of pieces of second point data located within the radius from the center point of the track from among the plurality of pieces of first point data. ### Claim 16 (depends on 13) The system according to claim 13, wherein the processor is configured to: select point data at a first location and point data at a second location among the plurality of pieces of second point data based on an angle formed by a straight line connecting each of the plurality of pieces of second point data from an origin and an X-axis in a vehicle coordinate system having a designated location in the vehicle as the origin; and update location information of the sensor fusion track based on information about the first location and the second location. ### Claim 17 (depends on 16) The system according to claim 16, wherein: the point data at the first location comprises point data having a smallest value as the angle among the plurality of pieces of second point data, and the point data at the second location comprises point data having a largest value as the angle among the plurality of pieces of second point data. ### Claim 18 (depends on 16) The system according to claim 16, wherein the processor is configured to: generate a two-dimensional equation representing a straight line connecting the first location and the second location; identify a first coordinate value of a first corner among corners of the sensor fusion track; determine a second coordinate value of a corner corresponding to the first corner among corners of the target vehicle based on the two-dimensional equation and the first coordinate value; and calculate a third coordinate value corresponding to a location of a midpoint of a rear bumper of the target vehicle, thereby updating location information of the sensor fusion track. ### Claim 19 (depends on 18) The system according to claim 18, wherein the processor is configured to: identify the second coordinate value having the same Y-axis coordinate value as a Y-axis coordinate value of the first coordinate value in the straight line connecting the first location and the second location; calculate a heading angle of the sensor fusion track to be updated based on a slope of the straight line connecting the first location and the second location; and calculate the third coordinate value based on a half value of a width of the track and the calculated heading angle. ### Claim 20 (depends on 19) The system according to claim 19, wherein: the heading angle is calculated by applying a slope value of the two-dimensional equation as a variable of an arc tangent function, an X-axis value of the third coordinate value is calculated based on a value calculated by applying the heading angle as a variable of a sine function, an X-axis value of the second coordinate value, and the half value of the width of the track, and a Y-axis value of the third coordinate value is calculated based on a value calculated by applying the heading angle as a variable of a cosine function, a Y-axis value of the second coordinate value, and the half value of the width of the track.