# US 10,369,994 B2 · Rear camera stub detection > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,369,994 B2 - Document ID: US10369994B2 - Status: Granted - Kind: B2 - Application number: 15/215,368 - Filed: 2016-07-20 - Published: 2018-01-25 - Granted: 2019-08-06 - Earliest claimed: 2016-07-20 - Assignees: Ford Global Technologies, LLC - Applicants: Ford Global Technologies, LLC - Inventors: Scott Vincent Myers; Alexandru Mihai Gurghian; Ashley Elizabeth Micks; Alexandro Walsh - Web page: https://patentgrep.com/patent/US10369994B2 ## CPC - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - B60W 30/0956: {the prediction being responsive to traffic or environmental parameters} - B60W 40/072: Curvature of the road - B60W 50/00: Details of control systems for road vehicle drive control not related to the control of a particular sub-unit {, e.g. process diagnostic or vehicle driver interfaces} - G01S 13/867: {Combination of radar systems with cameras} - G01S 15/86: Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection - G01S 15/89: for mapping or imaging - G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders - G06F 18/2413: based on distances to training or reference patterns - G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]} - G06V 10/764: using classification, e.g. of video objects - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - G06V 20/64: Three-dimensional [3D] objects - G08G 1/0962: having an indicator mounted inside the vehicle, e.g. giving voice messages - G08G 1/165: {for passive traffic, e.g. including static obstacles, trees} - G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection} - B60W 2050/0014: {Adaptive controllers} - B60W 2420/40: Photo, light or radio wave sensitive means, e.g. infrared sensors - B60W 2420/403: Image sensing, e.g. optical camera - B60W 2420/408: Radar; Laser, e.g. lidar - B60W 2420/54: Audio sensitive means, e.g. ultrasound - B60W 2552/20: Road profile, i.e. the change in elevation or curvature of a plurality of continuous road segments - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) ## Abstract A method for detecting stubs or intersecting roadways includes receiving perception data from at least two sensors. The at least two sensors include a rear facing camera of a vehicle and another sensor. The perception data includes information for a current roadway on which the vehicle is located. The method includes detecting, based on the perception data, an intersecting roadway connecting with the current roadway. The method also includes storing an indication of a location and a direction of the intersecting roadway with respect to the current roadway. ## Claims ### Claim 1 A method comprising: receiving perception data from at least two sensors of a vehicle that generate different data types, the at least two sensors comprising a rear facing camera and at least one of a light detection and ranging (LIDAR) system or a radar system, wherein the perception data comprises information for a current roadway on which the vehicle is located; generating fused data based on the perception data using weighted averages for the different data types; detecting, based on the fused data, an intersecting roadway connecting with the current roadway; and providing an indication of a location and a direction of the intersecting roadway with respect to the current roadway to an automated driving system or driver assistance system, whereby the automated driving system or driver assistance system actuates one or more vehicle controllers based on the indication. ### Claim 2 (depends on 1) The method of claim 1, wherein detecting the intersecting roadway comprises detecting one or more of: a gap in roadway markings, a break in a shoulder for the current roadway, or a variation or break in curb or barrier height. ### Claim 3 (depends on 1) The method of claim 1, wherein detecting the intersecting roadway comprises detecting using a deep neural network, and wherein generating the fused data further comprises modifying raw perception data to match a data format expected by the deep neural network. ### Claim 4 (depends on 1) The method of claim 1, wherein the at least two sensors comprise the rear facing camera, the LIDAR system, the radar system, an ultrasound sensing system, and an infrared camera system. ### Claim 5 (depends on 1) The method of claim 1, wherein the direction indicates a side of the current roadway on which the intersecting roadway is located. ### Claim 6 (depends on 1) The method of claim 1, further comprising storing the indication of the location and the direction of the intersecting roadway by uploading the indication to a remote storage location accessible over a network. ### Claim 7 (depends on 6) The method of claim 6, further comprising: determining a current location of the vehicle; retrieving drive history data from the remote storage location for the current location, wherein the drive history data indicates a location or direction of intersecting roadways near the current location; and broadcasting the location or direction of intersecting roadways near the current location to one or more vehicle controllers of an automated driving system or driving assistance system. ### Claim 8 (depends on 7) The method of claim 7, further comprising processing the location or direction of intersecting roadways to determine a route for the vehicle or detect a point of interest for the vehicle or a passenger. ### Claim 9 A system comprising: two or more sensors of a vehicle that generate different data types, the two or more sensors comprising a rear facing camera and at least one of a light detection and ranging (LIDAR) system or a radar system; a computing system comprising a processor configured to perform the following: receive perception data from the two or more sensors, wherein the perception data comprises information for a region behind the vehicle on a current roadway on which the vehicle is located; generate fused data based on the perception data using weighted averages for the different data types; detect, based on the fused data, an intersecting roadway connecting with the current roadway; determine a driving maneuver for the vehicle based on an indication of a location and a direction of the intersecting roadway with respect to the current roadway; and actuate one or more controllers of the vehicle to execute the driving maneuver. ### Claim 10 (depends on 9) The system of claim 9, wherein the computing system comprising the processor is configured to detect the intersecting roadway by detecting one or more of: a gap in roadway markings, a break in a shoulder for the current roadway, or a variation or break in curb or barrier height. ### Claim 11 (depends on 9) The system of claim 9, wherein the computing system comprising the processor is configured to detect the intersecting roadway by detecting using a deep neural network to process at least a portion of the perception data. ### Claim 12 (depends on 9) The system of claim 9, wherein the two or more sensors comprise the rear facing camera, the LIDAR system, the radar system, an ultrasound sensing system, and an infrared camera system. ### Claim 13 (depends on 9) The system of claim 9, wherein the computing system comprising the processor is configured to detect the direction, wherein the direction indicates a side of the current roadway on which the intersecting roadway is located. ### Claim 14 (depends on 9) The system of claim 9, wherein the computing system comprising the processor is configured to detect an intersecting roadway by detecting a driving surface that connects the current roadway to one or more of a driveway, parking lot, or a cross street. ### Claim 15 (depends on 9) The system of claim 9, wherein the computing system comprising the processor is configured to store the indication of the location and the direction of the intersecting roadway by uploading the indication to a remote storage location accessible over a network. ### Claim 16 (depends on 15) The system of claim 15, wherein the computing system comprising the processor is further configured to: determine a current location of the vehicle; and retrieve drive history data from the remote storage location for the current location, wherein the drive history data indicates a location or direction of intersecting roadways near the current location; and broadcast the location or direction of intersecting roadways near the current location to one or more vehicle controllers of an automated driving system or driving assistance system. ### Claim 17 (depends on 16) The system of claim 16, wherein the computing system comprising the processor is further configured to process the location or direction of the intersecting roadway to detect a point of interest for the vehicle or a passenger. ### Claim 18 Non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to: receive perception data from at least two sensors of a vehicle that generate different data types, the at least two sensors comprising a rear facing camera and at least one of a light detection and ranging (LIDAR) system or a radar system, wherein the perception data comprises information for a region behind the vehicle on a current roadway on which the vehicle is located; generate fused data based on the perception data using weighted averages for the different data types; detect, based on the fused data, an intersecting roadway connecting with the current roadway; and provide an indication of a location and a direction of the intersecting roadway with respect to the current roadway to an automated driving system or driver assistance system, whereby the automated driving system or driver assistance system actuates one or more vehicle controllers based on the indication. ### Claim 19 (depends on 18) The non-transitory computer readable storage media of claim 18, wherein the instructions cause the one or more processors to detect the intersecting roadway by detecting one or more of: a gap in roadway markings, a break in a shoulder for the current roadway, or a variation or break in curb or barrier height. ### Claim 20 (depends on 18) The non-transitory computer readable storage media of claim 18, wherein the instructions further cause the one or more processors to: store the indication of the location and direction by uploading to a remote storage location accessible over a network, determine a current location of the vehicle; retrieve drive history data from the remote storage location for the current location, wherein the drive history data indicates a location or direction of intersecting roadways near the current location; and broadcast the location or direction of intersecting roadways near the current location to one or more vehicle controllers of an automated driving system or driving assistance system.