# US 12,602,935 B2 · Driving assistance device and driving assistance method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,935 B2 - Document ID: US12602935B2 - Status: Granted - Kind: B2 - Application number: 18/562,754 - Filed: 2022-02-04 - Published: 2024-08-15 - Granted: 2026-04-14 - Earliest claimed: 2021-07-01 - Assignees: Hitachi Astemo, Ltd. - Applicants: Hitachi Astemo, Ltd. - Inventors: Takahiro Sakai; Morihiko Sakano; Hideyuki Kume; Hitoshi Hayakawa; Tatsuhiko Monji - Web page: https://patentgrep.com/patent/US12602935B2 ## CPC - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - G08G 1/096775: {where the origin of the information is a central station} - B60W 30/12: Lane keeping - G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components - G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model - G08G 1/09626: {where the origin of the information is within the own vehicle, e.g. a local storage device, digital map} - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2552/53: Road markings, e.g. lane marker or crosswalk - B60W 2556/40: High definition maps - G06V 10/806: {of extracted features} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT ## Abstract A driving assistance device that recognizes a lane marking around an own vehicle with at least one sensor includes a lane marking detection unit that detects lane marking information from data acquired by the sensor, a lane marking type candidate estimation unit that recognizes a lane marking type from the detected lane marking information and calculates a likelihood of each lane marking type, a lane marking type branch point estimation unit that detects a lane marking type branch point that is a point at which a likelihood of the lane marking type changes with respect to an advancing direction of the own vehicle, and distinguishes regions before and after the lane marking type branch point as a first region and a second region, and a lane marking information integration unit that, in a case where a lane marking of the first region and a lane marking of the second region have the same lane marking type, integrates the lane marking of the first region and the lane marking of the second region into a continuously recognizable lane marking. ## Claims ### Claim 1 A driving assistance device that recognizes a lane marking around an own vehicle with at least one sensor, the driving assistance device comprising: a lane marking detection unit that detects lane marking information from data acquired by the sensor; a lane marking type candidate estimation unit that recognizes a lane marking type from the detected lane marking information and calculates a likelihood of each lane marking type; a lane marking type branch point estimation unit that detects a lane marking type branch point that is a point at which a likelihood of the lane marking type changes with respect to an advancing direction of the own vehicle, and distinguishes regions before and after the lane marking type branch point as a first region and a second region; and a lane marking information integration unit that, in a case where a lane marking of the first region and a lane marking of the second region have a same lane marking type, integrates the lane marking of the first region and the lane marking of the second region into a continuously recognizable lane marking. ### Claim 2 (depends on 1) The driving assistance device according to claim 1, further comprising a lane marking integration necessity determination unit that determines necessity of integration of the lane marking of the first region and the lane marking of the second region, wherein in a case where the lane marking integration necessity determination unit determines that integration is necessary, the lane marking information integration unit integrates the lane marking of the first region and the lane marking of the second region into a continuously recognizable lane marking. ### Claim 3 (depends on 1) The driving assistance device according to claim 1, further comprising: a scene understanding unit that estimates a situation around the vehicle from the data acquired by the sensor; and a lane marking reliability estimation unit that estimates a reliability of the lane marking type determined for each of the sensors based on an output of the scene understanding unit and determines a reliability of the lane marking for each of the sensors, wherein the lane marking type candidate estimation unit refers to the estimated reliability, and determines lane marking information of a lane marking type candidate for each of the sensors based on an edge of the lane marking by using at least one of luminance and reflection intensity acquired by the sensor. ### Claim 4 (depends on 1) The driving assistance device according to claim 1, wherein the lane marking type candidate estimation unit calculates a likelihood of each lane marking type by comparing luminance information of a lane marking sampled at regular intervals in an advancing direction of the own vehicle with a lane marking shape model obtained by modeling the luminance information. ### Claim 5 (depends on 1) The driving assistance device according to claim 1, wherein the lane marking information integration unit associates the lane markings by using the lane marking type, and integrates the lane marking information. ### Claim 6 (depends on 1) The driving assistance device according to claim 1, wherein the lane marking type branch point estimation unit divides the lane marking into regions before and after the lane marking type branch point and manages the lane marking information for each lane marking type. ### Claim 7 (depends on 1) The driving assistance device according to claim 1, wherein the lane marking type branch point estimation unit detects a lane marking branch point based on a ratio of the likelihood of the lane marking type in a case where a plurality of lane marking types are assigned. ### Claim 8 (depends on 3) The driving assistance device according to claim 3, wherein the scene understanding unit sets a reliability of a lane marking based on a disturbance factor, a sensor type, and a lane marking type. ### Claim 9 (depends on 3) The driving assistance device according to claim 3, wherein the lane marking information integration unit integrates a lane marking of a type of which a likelihood is calculated for each sensor by using a reliability for each sensor based on a situation around the vehicle estimated by the scene understanding unit. ### Claim 10 (depends on 3) The driving assistance device according to claim 3, wherein the lane marking information integration unit integrates a lane marking of a type of which a likelihood is calculated for each sensor by using a likelihood of a lane marking type that is likely to appear in a situation around the vehicle estimated by the scene understanding unit. ### Claim 11 (depends on 1) The driving assistance device according to claim 1, wherein the lane marking type candidate estimation unit recognizes a type of a lane marking from the detected lane marking information and map information, and the lane marking type branch point estimation unit detects a lane marking type branch point that is a point at which a likelihood of the lane marking type changes in a traveling direction of the own vehicle by using the map information. ### Claim 12 (depends on 1) The driving assistance device according to claim 1, wherein the lane marking information integration unit integrates the lane markings by using the number of lanes and a road width acquired from map information. ### Claim 13 (depends on 1) The driving assistance device according to claim 1, wherein the lane marking information integration unit integrates a lane marking of a type of which a likelihood is calculated for each sensor by using a likelihood of a lane marking type that is likely to appear at a position estimated from map information. ### Claim 14 (depends on 1) The driving assistance device according to claim 1, wherein the lane marking information integration unit registers information regarding a position, a type, and a branch point of the integrated lane marking in map information. ### Claim 15 A driving assistance method executed by a driving assistance device that recognizes a lane marking around an own vehicle with at least one sensor, wherein the driving assistance device includes an arithmetic device that executes predetermined processing and a storage device that is connected to the arithmetic device, the driving assistance method comprising: a lane marking detection procedure in which the arithmetic device detects lane marking information from data acquired by the sensor; a lane marking type candidate estimation procedure in which the arithmetic device recognizes a lane marking type from the detected lane marking information and calculates a likelihood of each lane marking type; a lane marking branch point estimation procedure in which the arithmetic device detects a lane marking type branch point that is a point at which a likelihood of the lane marking type changes with respect to an advancing direction of the own vehicle, and distinguishes regions before and after the lane marking type branch point as a first region and a second region; and a lane marking information integration procedure in which, in a case where a lane marking of the first region and a lane marking of the second region have a same lane marking type, the arithmetic device integrates the lane marking of the first region and the lane marking of the second region into a continuously recognizable lane marking.