# US 12,118,806 B2 · Vehicular imaging system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,806 B2 - Document ID: US12118806B2 - Status: Granted - Kind: B2 - Application number: 17/659,508 - Filed: 2022-04-18 - Published: 2022-08-04 - Granted: 2024-10-15 - Earliest claimed: 2004-12-23 - Assignees: Magna Electronics Inc. - Applicants: Magna Electronics Inc. - Inventors: Michael J. Higgins-Luthman - Web page: https://patentgrep.com/patent/US12118806B2 ## CPC - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - B60Q 9/008: {for anti-collision purposes} - B60R 1/12: Mirror assemblies combined with other articles, e.g. clocks - B60R 1/23: with a predetermined field of view - B60W 40/04: Traffic conditions - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 20/584: {of vehicle lights or traffic lights} - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 7/181: {for receiving images from a plurality of remote sources} - H04N 23/80: Camera processing pipelines; Components thereof - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - H04N 25/76: Addressed sensors, e.g. MOS or CMOS sensors - B60R 2001/1253: {with cameras, video cameras or video screens} - B60R 2300/105: using multiple cameras - B60R 2300/30: characterised by the type of image processing - B60R 2300/402: Image calibration - B60R 2300/802: for monitoring and displaying vehicle exterior blind spot views - B60R 2300/804: for lane monitoring - B60R 2300/806: for aiding parking - B60R 2300/8066: for monitoring rearward traffic - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A vehicular imaging system includes a camera operable to capture image data. The camera is configured for attachment at an upper windshield area of an in-cabin side of a windshield of a vehicle. An image processor is operable for processing image data captured by the camera. Captured image data is processed by the image processor for a collision avoidance system of the equipped vehicle. Captured image data is processed by the image processor for a lane departure warning system of the equipped vehicle. Responsive at least in part to processing by the image processor of captured image data, shadows viewed by the camera are detected. The vehicular imaging system determines misalignment of the camera responsive at least in part to processing by the image processor of captured image data. ## Claims ### Claim 1 A vehicular imaging system comprising: a camera comprising a CMOS imaging sensor having a plurality of photo-sensing pixels; wherein the camera is operable to capture image data; wherein the camera is configured for attachment at an upper windshield area of an in-cabin side of a windshield of a vehicle equipped with the vehicular imaging system; wherein, with the camera attached at the windshield of the equipped vehicle, the camera views forward of the equipped vehicle through an area of the windshield that is cleaned by a windshield wiper of the equipped vehicle when the windshield wiper is activated; wherein, with the camera attached at the windshield of the equipped vehicle, image data captured by the camera is provided to an image processor; wherein the image processor is operable to process image data captured by the camera and provided to the image processor; wherein image data captured by the camera and provided to the image processor is processed by the image processor for a collision avoidance system of the equipped vehicle; wherein image data captured by the camera and provided to the image processor is processed by the image processor for a lane departure warning system of the equipped vehicle; wherein, at least in part via processing by the image processor of image data captured by the camera and provided to the image processor, shadows viewed by the camera are detected; and wherein the vehicular imaging system determines misalignment of the camera at least in part via processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 2 (depends on 1) The vehicular imaging system of claim 1, wherein the vehicular imaging system determines another vehicle responsive at least in part to detecting, via processing by the image processor of image data captured by the camera and provided to the image processor, wheel of the other vehicle. ### Claim 3 (depends on 2) The vehicular imaging system of claim 2, wherein the vehicular imaging system determines the other vehicle at least in part responsive to determination, via processing by the image processor of image data captured by the camera and provided to the image processor, of speed of the other vehicle. ### Claim 4 (depends on 1) The vehicular imaging system of claim 1, wherein, responsive at least in part to processing by the image processor of image data captured by the camera and provided to the image processor, a driver of the equipped vehicle is alerted to presence of a bicycle. ### Claim 5 (depends on 1) The vehicular imaging system of claim 1, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 6 (depends on 5) The vehicular imaging system of claim 5, wherein use of edge detection comprises detection of vertical edges of other vehicles viewed by the camera. ### Claim 7 (depends on 6) The vehicular imaging system of claim 6, wherein the detected vertical edges are used at least in part in detection of the vertical edges of the other vehicles by the vehicular imaging system. ### Claim 8 (depends on 7) The vehicular imaging system of claim 7, wherein the vehicular imaging system is operable in a daytime mode and a nighttime mode. ### Claim 9 (depends on 8) The vehicular imaging system of claim 8, wherein the vehicular imaging system switches between the daytime mode and the nighttime mode in response to an ambient light level at the equipped vehicle. ### Claim 10 (depends on 1) The vehicular imaging system of claim 1, wherein, at least in part via processing by the image processor of image data captured by the camera and provided to the image processor, the vehicular imaging system distinguishes a headlight from other light sources viewed by the camera. ### Claim 11 (depends on 1) The vehicular imaging system of claim 1, wherein image data captured by the camera is provided to the image processor as digital data. ### Claim 12 (depends on 1) The vehicular imaging system of claim 1, wherein image data captured by the camera and provided to the image processor is processed by the image processor for an adaptive cruise control system of the equipped vehicle. ### Claim 13 (depends on 1) The vehicular imaging system of claim 1, wherein the vehicular imaging system determines another vehicle at least in part via processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 14 (depends on 13) The vehicular imaging system of claim 13, wherein, in detecting the other vehicle and at least in part via processing by the image processor of image data captured by the camera and provided to the image processor, shadows viewed by the camera are detected and distinguished from the other vehicle. ### Claim 15 (depends on 13) The vehicular imaging system of claim 13, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor, and wherein, in detecting the other vehicle viewed by the camera, edges that are indicative of a self-shadow of the equipped vehicle are ignored during processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 16 (depends on 13) The vehicular imaging system of claim 13, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor, and wherein, in detecting the other vehicle viewed by the camera, edges that are indicative of shadows cast on a road the equipped vehicle is traveling along are ignored during processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 17 (depends on 13) The vehicular imaging system of claim 13, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor, and wherein, in detecting the other vehicle viewed by the camera, edges that are indicative of a shadow cast by a tree on a road the equipped vehicle is traveling along are ignored during processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 18 (depends on 1) The vehicular imaging system of claim 1, wherein the vehicular imaging system detects a bicycle viewed by the camera at least in part via processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 19 (depends on 18) The vehicular imaging system of claim 18, wherein the vehicular imaging system detects the bicycle responsive at least in part to determining, via processing by the image processor of image data captured by the camera and provided to the image processor, a wheel of the bicycle. ### Claim 20 (depends on 19) The vehicular imaging system of claim 19, wherein the vehicular imaging system determines the wheel of the bicycle by detecting vertical edges of the bicycle via processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 21 (depends on 18) The vehicular imaging system of claim 18, wherein the vehicular imaging system utilizes edge detection in processing image data captured by the camera and provided to the image processor to detect a wheel of the bicycle viewed by the camera. ### Claim 22 (depends on 18) The vehicular imaging system of claim 18, wherein the vehicular imaging system utilizes edge detection in processing image data captured by the camera and provided to the image processor to detect a bicyclist on the bicycle viewed by the camera. ### Claim 23 (depends on 18) The vehicular imaging system of claim 18, wherein the vehicular imaging system determines the bicycle at least in part responsive to determination, via processing by the image processor of image data captured by the camera and provided to the image processor, of speed of the bicycle. ### Claim 24 (depends on 18) The vehicular imaging system of claim 18, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor, and wherein, in detecting the bicycle viewed by the camera, edges that are indicative of a self-shadow of the equipped vehicle are ignored during processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 25 A vehicular imaging system comprising: a camera comprising a CMOS imaging sensor having a plurality of photo-sensing pixels; wherein the camera is operable to capture image data; wherein the camera is configured for attachment at an upper windshield area of an in-cabin side of a windshield of a vehicle equipped with the vehicular imaging system; wherein, with the camera attached at the windshield of the equipped vehicle, the camera views forward of the equipped vehicle through an area of the windshield that is cleaned by a windshield wiper of the equipped vehicle when the windshield wiper is activated; wherein, with the camera attached at the windshield of the equipped vehicle, image data captured by the camera is provided to an image processor; wherein the image processor is operable to process image data captured by the camera and provided to the image processor; wherein image data captured by the camera and provided to the image processor is processed by the image processor for a collision avoidance system of the equipped vehicle; wherein image data captured by the camera and provided to the image processor is processed by the image processor for a lane departure warning system of the equipped vehicle; wherein image data captured by the camera and provided to the image processor is processed by the image processor for an adaptive cruise control system of the equipped vehicle; wherein, at least in part via processing by the image processor of image data captured by the camera and provided to the image processor, the vehicular imaging system detects an object viewed by the camera; wherein, in detecting the object viewed by the camera, a self-shadow of the equipped vehicle is ignored during processing by the image processor of image data captured by the camera and provided to the image processor; and wherein the vehicular imaging system determines misalignment of the camera at least in part via processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 26 (depends on 25) The vehicular imaging system of claim 25, wherein the vehicular imaging system determines that the detected object comprises another vehicle responsive at least in part to detecting, via processing by the image processor of image data captured by the camera and provided to the image processor, a wheel of the other vehicle. ### Claim 27 (depends on 26) The vehicular imaging system of claim 26, wherein the vehicular imaging system determines that the detected object comprises another vehicle at least in part responsive to determination, via processing by the image processor of image data captured by the camera and provided to the image processor, of speed of the object. ### Claim 28 (depends on 25) The vehicular imaging system of claim 25, wherein, responsive at least in part to processing by the image processor of image data captured by the camera and provided to the image processor, a driver of the equipped vehicle is alerted to presence of the detected object. ### Claim 29 (depends on 25) The vehicular imaging system of claim 25, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 30 (depends on 29) The vehicular imaging system of claim 29, wherein use of edge detection comprises detection of vertical edges of objects viewed by the camera. ### Claim 31 (depends on 30) The vehicular imaging system of claim 30, wherein the detected vertical edges are used at least in part in detection of the object by the vehicular imaging system. ### Claim 32 (depends on 25) The vehicular imaging system of claim 25, wherein, at least in part via processing by the image processor of image data captured by the camera and provided to the image processor, the vehicular imaging system distinguishes a headlight from other light sources viewed by the camera. ### Claim 33 (depends on 25) The vehicular imaging system of claim 25, wherein the vehicular imaging system is operable in a daytime mode and a nighttime mode, and wherein the vehicular imaging system switches between the daytime mode and the nighttime mode in response to an ambient light level at the equipped vehicle. ### Claim 34 (depends on 25) The vehicular imaging system of claim 25, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor, and wherein, in detecting the other vehicle viewed by the camera, edges that are indicative of shadows cast on a road the equipped vehicle is traveling along are ignored during processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 35 (depends on 25) The vehicular imaging system of claim 25, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor, and wherein, in detecting the object viewed by the camera, edges that are indicative of a shadow cast by a tree on a road the equipped vehicle is traveling along are ignored during processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 36 (depends on 25) The vehicular imaging system of claim 25, wherein the detected object comprises another vehicle, and wherein, in detecting the other vehicle and at least in part via processing by the image processor of image data captured by the camera and provided to the image processor, shadows viewed by the camera are discerned and distinguished from the other vehicle. ### Claim 37 (depends on 25) The vehicular imaging system of claim 25, wherein the vehicular imaging system determines that the detected object comprises a bicycle. ### Claim 38 A vehicular imaging system comprising: a camera comprising a CMOS imaging sensor having a plurality of photo-sensing pixels; wherein the camera is operable to capture image data; wherein the camera is configured for attachment at an upper windshield area of an in-cabin side of a windshield of a vehicle equipped with the vehicular imaging system; wherein, with the camera attached at the windshield of the equipped vehicle, the camera views forward of the equipped vehicle through an area of the windshield that is cleaned by a windshield wiper of the equipped vehicle when the windshield wiper is activated; wherein, with the camera attached at the windshield of the equipped vehicle, image data captured by the camera is provided to an image processor; wherein the image processor is operable for processing image data captured by the camera and provided to the image processor; wherein image data captured by the camera and provided to the image processor is processed by the image processor for a collision avoidance system of the equipped vehicle; wherein image data captured by the camera and provided to the image processor is processed by the image processor for a lane departure warning system of the equipped vehicle; wherein, at least in part via processing by the image processor of image data captured by the camera and provided to the image processor, shadows viewed by the camera are detected; wherein, at least in part via processing by the image processor of image data captured by the camera and provided to the image processor, the vehicular imaging system detects a bicycle viewed by the camera; wherein, in detecting the bicycle viewed by the camera, shadows cast on a road the equipped vehicle is traveling along are ignored during processing by the image processor of image data captured by the camera and provided to the image processor; wherein the vehicular imaging system determines misalignment of the camera at least in part via processing by the image processor of image data captured by the camera and provided to the image processor; and wherein, responsive at least in part to processing by the image processor of image data captured by the camera and provided to the image processor, a driver of the equipped vehicle is alerted to presence of the detected bicycle. ### Claim 39 (depends on 38) The vehicular imaging system of claim 38, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 40 (depends on 39) The vehicular imaging system of claim 39, wherein use of edge detection comprises detection of vertical edges of objects viewed by the camera. ### Claim 41 (depends on 40) The vehicular imaging system of claim 40, wherein the detected vertical edges are used at least in part in detection of the bicycle by the vehicular imaging system. ### Claim 42 (depends on 38) The vehicular imaging system of claim 38, wherein, at least in part via processing by the image processor of image data captured by the camera and provided to the image processor, the vehicular imaging system distinguishes a headlight from other light sources viewed by the camera. ### Claim 43 (depends on 38) The vehicular imaging system of claim 38, wherein the vehicular imaging system is operable in a daytime mode and a nighttime mode, and wherein the vehicular imaging system switches between the daytime mode and the nighttime mode in response to an ambient light level at the equipped vehicle. ### Claim 44 (depends on 38) The vehicular imaging system of claim 38, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor, and wherein, in detecting the bicycle viewed by the camera, edges that are indicative of a shadow cast by a tree on a road the equipped vehicle is traveling along are ignored during processing by the image processor of image data captured by the camera and provided to the image processor. ### Claim 45 (depends on 38) The vehicular imaging system of claim 38, wherein the vehicular imaging system detects the bicycle responsive at least in part to determining, via processing by the image processor of image data captured by the camera and provided to the image processor, a wheel of the bicycle. ### Claim 46 (depends on 45) The vehicular imaging system of claim 45, wherein the vehicular imaging system utilizes edge detection in processing by the image processor of image data captured by the camera and provided to the image processor, and wherein the vehicular imaging system determines a wheel of the bicycle by detecting vertical edges of the bicycle. ### Claim 47 (depends on 38) The vehicular imaging system of claim 38, wherein the vehicular imaging system utilizes edge detection in processing image data captured by the camera and provided to the image processor to detect a wheel of the bicycle viewed by the camera. ### Claim 48 (depends on 38) The vehicular imaging system of claim 38, wherein the vehicular imaging system utilizes edge detection in processing image data captured by the camera and provided to the image processor to detect a bicyclist on the bicycle viewed by the camera. ### Claim 49 (depends on 38) The vehicular imaging system of claim 38, wherein the vehicular imaging system detects the bicycle at least in part responsive to determination, via processing by the image processor of image data captured by the camera and provided to the image processor, of speed of the bicycle.