# US 11,308,720 B2 · Vehicular imaging system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,720 B2 - Document ID: US11308720B2 - Status: Granted - Kind: B2 - Application number: 16/714,908 - Filed: 2019-12-16 - Published: 2020-04-16 - Granted: 2022-04-19 - Earliest claimed: 2004-12-23 - Assignees: Magna Electronics Inc. - Applicants: Magna Electronics Inc. - Inventors: Michael J. Higgins-Luthman - Web page: https://patentgrep.com/patent/US11308720B2 ## CPC - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - B60Q 9/008: {for anti-collision purposes} - B60R 1/12: Mirror assemblies combined with other articles, e.g. clocks - B60R 1/23: with a predetermined field of view - B60W 40/04: Traffic conditions - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 20/584: {of vehicle lights or traffic lights} - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 7/181: {for receiving images from a plurality of remote sources} - H04N 23/80: Camera processing pipelines; Components thereof - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - H04N 25/76: Addressed sensors, e.g. MOS or CMOS sensors - B60R 2001/1253: {with cameras, video cameras or video screens} - B60R 2300/105: using multiple cameras - B60R 2300/30: characterised by the type of image processing - B60R 2300/402: Image calibration - B60R 2300/802: for monitoring and displaying vehicle exterior blind spot views - B60R 2300/804: for lane monitoring - B60R 2300/806: for aiding parking - B60R 2300/8066: for monitoring rearward traffic - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A vehicular imaging system includes an imaging device comprising an array of photo-sensing pixels and a control having an image processor operable for processing image data captured by the imaging device. The control utilizes edge detection in processing captured image data by the image processor for detecting objects present exterior of the vehicle. Responsive at least in part to processing at the control of captured image data by the image processor, the control detects an object of interest present in the field of view of the imaging device. In detecting the object of interest present in the field of view of the imaging device and responsive at least in part to processing at the control of captured image data by the image processor, shadows present in the field of view of the imaging device are discerned and distinguished from the object of interest. ## Claims ### Claim 1 An imaging system for detecting objects exterior of a vehicle, said imaging system comprising: an imaging device comprising an array of photo-sensing pixels; wherein said imaging device is operable to capture image data; wherein, with said imaging device attached at a vehicle equipped with said imaging system, said imaging device has a field of view exterior of the equipped vehicle; a control disposed at the equipped vehicle; said control comprising an image processor operable for processing image data captured by said imaging device; wherein said control utilizes edge detection in processing captured image data for detecting objects present exterior of the equipped vehicle within the field of view of said imaging device; wherein, responsive at least in part to processing at said control of captured image data by said image processor, said imaging system detects an object of interest present in the field of view of said imaging device; wherein the object of interest present in the field of view of said imaging device comprises a bicycle; and wherein, in detecting the object of interest present in the field of view of said imaging device and responsive at least in part to processing at said control of captured image data by said image processor, shadows present in the field of view of said imaging device are discerned and distinguished from the object of interest. ### Claim 2 (depends on 1) The imaging system of claim 1, wherein said imaging system determines that the object of interest present in the field of view of said imaging device comprises a bicycle responsive at least in part to processing at said control of captured image data by said image processor determining a wheel of the bicycle. ### Claim 3 (depends on 1) The imaging system of claim 1, wherein, responsive at least in part to processing at said control of captured image data by said image processor, a driver of the equipped vehicle is alerted to presence of the object of interest. ### Claim 4 (depends on 1) The imaging system of claim 1, wherein the field of view of said imaging device is at least one selected from the group consisting of (j) rearward of the equipped vehicle and (ii) sideward of the equipped. ### Claim 5 (depends on 1) The imaging system of claim 1, wherein, responsive at least in part to processing at said control of captured image data by said image processor, said imaging system distinguishes a headlight from other light sources present in the field of view of said imaging device. ### Claim 6 (depends on 1) The imaging system of claim 1, wherein said imaging device is configured to attach at a side of the equipped vehicle, and wherein, with said imaging device attached at the side of the equipped vehicle, a portion of the side of the equipped vehicle is present in the field of view of said imaging device. ### Claim 7 (depends on 1) The imaging system of claim 1, wherein said imaging system determines that the object of interest present in the field of view of said imaging device comprises a bicycle at least in part responsive to determination, at least in part via processing at said control of captured image data by said image processor, of speed of the object of interest. ### Claim 8 (depends on 1) The imaging system of claim 1, wherein use of edge detection at said control comprises detection of vertical edges of objects present in the field of view of said imaging device. ### Claim 9 (depends on 8) The imaging system of claim 8, wherein the detected vertical edges are used at least in part in detection of the object of interest at said control. ### Claim 10 (depends on 1) The imaging system of claim 1, wherein said imaging system determines misalignment of said imaging device responsive at least in part to processing at said control of captured image data by said image processor. ### Claim 11 (depends on 10) The imaging system of claim 10, wherein said imaging system determines a wheel of the bicycle by detecting vertical edges of the bicycle. ### Claim 12 (depends on 1) The imaging system of claim 1, wherein said control is operable in a daytime mode and a nighttime mode. ### Claim 13 (depends on 12) The imaging system of claim 12, wherein said control switches between the daytime mode and the nighttime mode in response to an ambient light level at the equipped vehicle. ### Claim 14 (depends on 1) The imaging system of claim 1, wherein image data captured by said imaging device is provided to said control as a digital data stream. ### Claim 15 (depends on 1) The imaging system of claim 1, wherein said imaging device is configured to attach at a side of the equipped vehicle, and wherein, with said imaging device attached at the side of the equipped vehicle, a driver of the equipped vehicle is alerted that the object of interest is present exterior of the equipped vehicle in response to at least one selected from the group consisting of (a) the driver of the equipped vehicle actuating a turn signal toward the side of the equipped vehicle at which the object of interest is present and (b) the driver of the equipped vehicle steering the equipped vehicle toward the side of the equipped vehicle at which the object of interest is present. ### Claim 16 (depends on 15) The imaging system of claim 15, wherein, with said imaging device attached at the side of the equipped vehicle, a portion of the side of the equipped vehicle is present in the field of view of said imaging device, and wherein said imaging system determines that said portion of the equipped vehicle is not an object of interest responsive at least in part to processing at said control of captured image data by said image processor. ### Claim 17 (depends on 15) The imaging system of claim 15, wherein, with said imaging device attached at the side of the equipped vehicle, a portion of the side of the equipped vehicle is present in the field of view of said imaging device. ### Claim 18 (depends on 17) The imaging system of claim 17, wherein said imaging device is part of an exterior rearview mirror assembly that attaches at the side of the equipped vehicle. ### Claim 19 (depends on 1) The imaging system of claim 1, wherein said imaging device is configured to attach at a side of the equipped vehicle, and wherein, with said imaging device attached at the side of the equipped vehicle, said imaging device has a field of view forward of the equipped vehicle. ### Claim 20 (depends on 1) The imaging system of claim 1, wherein, in detecting the object of interest present in the field of view of said imaging device, an algorithmically executed filter is utilized that ignores detected edges that are not indicative of the object of interest. ### Claim 21 (depends on 1) The imaging system of claim 1, wherein a video display screen is located in the interior cabin of the equipped vehicle and is viewable by a driver of the equipped vehicle, and wherein video images are displayed by said video display screen that are derived, at least in part, from image data captured by said imaging device. ### Claim 22 (depends on 21) The imaging system of claim 21, wherein said video display screen is disposed at an interior rearview mirror assembly of the equipped vehicle. ### Claim 23 (depends on 1) The imaging system of claim 1, wherein, in detecting the object of interest present in the field of view of said imaging device, edges that are indicative of a self-shadow of the equipped vehicle are ignored. ### Claim 24 (depends on 1) The imaging system of claim 1, wherein, in detecting the object of interest present in the field of view of said imaging device, edges that are indicative of a shadow cast by a tree on a road the equipped vehicle is traveling along are ignored. ### Claim 25 (depends on 1) The imaging system of claim 1, wherein said control utilizes edge detection in processing captured image data to detect a wheel of the bicycle present in the field of view of said imaging device. ### Claim 26 (depends on 1) The imaging system of claim 1, wherein said control utilizes edge detection in processing captured image data to detect a bicyclist on the bicycle present in the field of view of said imaging device. ### Claim 27 An imaging system for detecting objects exterior of a vehicle, said imaging system comprising: an imaging device comprising a CMOS array of photo-sensing pixels; wherein said imaging device is operable to capture image data; wherein said imaging device is disposed in a windshield electronics module that is configured to attach at an upper windshield area of an in-cabin side of a windshield of a vehicle equipped with said imaging system; wherein, with the windshield electronics module attached at the windshield of the equipped vehicle, said imaging device has a field of view forward of the equipped vehicle through an area of the windshield that is cleaned by a windshield wiper of the equipped vehicle when the windshield wiper is activated; a control disposed at the equipped vehicle; said control comprising an image processor operable for processing image data captured by said imaging device; wherein said control utilizes edge detection in processing captured image data for detecting objects present exterior of the equipped vehicle within the field of view of said imaging device; wherein captured image data is processed at said control for a collision avoidance system of the equipped vehicle; wherein, responsive at least in part to processing at said control of captured image data by said image processor, said imaging system distinguishes a headlight from other light sources present in the field of view of said imaging device; wherein, responsive at least in part to processing at said control of captured image data by said image processor, said imaging system detects an object of interest present in the field of view of said imaging device; and wherein, in detecting the object of interest present in the field of view of said imaging device and responsive at least in part to processing at said control of captured image data by said image processor, shadows present in the field of view of said imaging device are discerned and distinguished from the object of interest. ### Claim 28 (depends on 27) The imaging system of claim 27, wherein said imaging system determines that the object of interest present in the field of view of said imaging device comprises a bicycle. ### Claim 29 (depends on 28) The imaging system of claim 28, wherein said imaging system determines that the object of interest present in the field of view of said imaging device comprises a bicycle responsive at least in part to processing at said control of captured image data by said imaging system detecting a wheel of the bicycle. ### Claim 30 (depends on 28) The imaging system of claim 28, wherein, responsive at least in part to processing at said control of captured image data by said image processor, a driver of the equipped vehicle is alerted to presence of the object of interest. ### Claim 31 (depends on 28) The imaging system of claim 28, wherein said control utilizes edge detection in processing captured image data to detect a bicyclist on the bicycle present in the field of view of said imaging device. ### Claim 32 (depends on 27) The imaging system of claim 27, wherein said imaging system determines that the object of interest present in the field of view of said imaging device comprises another vehicle. ### Claim 33 (depends on 32) The imaging system of claim 32, wherein said imaging system determines that the object of interest present in the field of view of said imaging device comprises another vehicle responsive at least in part to processing at said control of captured image data by said imaging system detecting a wheel of the other vehicle. ### Claim 34 (depends on 32) The imaging system of claim 32, wherein, responsive at least in part to processing at said control of captured image data by said image processor, a driver of the equipped vehicle is alerted to presence of the object of interest. ### Claim 35 (depends on 27) The imaging system of claim 27, wherein said imaging system detects the object of interest present in the field of view of said imaging device at least in part responsive to determination, at least in part via processing at said control of captured image data by said image processor, of speed of the object of interest. ### Claim 36 (depends on 27) The imaging system of claim 27, wherein use of edge detection at said control comprises detection of vertical edges of objects present in the field of view of said imaging device. ### Claim 37 (depends on 36) The imaging system of claim 36, wherein the detected vertical edges are used at least in part in detection of the object of interest at said control. ### Claim 38 (depends on 27) The imaging system of claim 27, wherein said imaging system determines misalignment of said imaging device responsive at least in part to processing at said control of captured image data by said image processor. ### Claim 39 (depends on 27) The imaging system of claim 27, wherein said control is operable in a daytime mode and a nighttime mode. ### Claim 40 (depends on 39) The imaging system of claim 39, wherein said control switches between the daytime mode and the nighttime mode in response to an ambient light level at the equipped vehicle. ### Claim 41 (depends on 27) The imaging system of claim 27, wherein image data captured by said imaging device is provided to said control as a digital data stream. ### Claim 42 (depends on 27) The imaging system of claim 27, wherein, in detecting the object of interest present in the field of view of said imaging device, an algorithmically executed filter is utilized that ignores detected edges that are not indicative of the object of interest. ### Claim 43 (depends on 27) The imaging system of claim 27, wherein, in detecting the object of interest present in the field of view of said imaging device, edges that are indicative of a shadow cast by a tree on a road the equipped vehicle is traveling along are ignored. ### Claim 44 (depends on 27) The imaging system of claim 27, wherein captured image data is processed at said control for a lane departure warning system of the equipped vehicle. ### Claim 45 (depends on 44) The imaging system of claim 44, wherein captured image data is processed at said control for an adaptive cruise control system of the equipped vehicle. ### Claim 46 (depends on 27) The imaging system of claim 27, wherein said imaging system determines that the object of interest present in the field of view of said imaging device comprises another vehicle travelling in a traffic lane adjacent to a traffic lane that the equipped vehicle is travelling along. ### Claim 47 (depends on 46) The imaging system of claim 46, wherein, in determining that the object of interest present in the field of view of said imaging device comprises another vehicle, shadows cast by objects outside of the adjacent traffic lane are discerned and distinguished from objects present in the adjacent traffic lane. ### Claim 48 An imaging system for detecting objects exterior of a vehicle, said imaging system comprising: an imaging device comprising a CMOS array of photo-sensing pixels; wherein said imaging device is operable to capture image data; wherein said imaging device is disposed in a windshield electronics module that is configured to attach at an upper windshield area of an in-cabin side of a windshield of a vehicle equipped with said imaging system; wherein, with the windshield electronics module attached at the windshield of the equipped vehicle, said imaging device has a field of view forward of the equipped vehicle through an area of the windshield that is cleaned by a windshield wiper of the equipped vehicle when the windshield wiper is activated; a control disposed at the equipped vehicle; said control comprising an image processor operable for processing image data captured by said imaging device; wherein said control utilizes edge detection in processing captured image data for detecting objects present exterior of the equipped vehicle within the field of view of said imaging device; wherein captured image data is processed at said control for a collision avoidance system of the equipped vehicle; wherein captured image data is processed at said control for a lane departure warning system of the equipped vehicle; wherein, responsive at least in part to processing at said control of captured image data by said image processor, said imaging system distinguishes a headlight from other light sources present in the field of view of said imaging device; wherein, responsive at least in part to processing at said control of captured image data by said image processor, said imaging system detects an object of interest present in the field of view of said imaging device; wherein the object of interest present in the field of view of said imaging device comprises another vehicle; and wherein, in detecting the other vehicle present in the field of view of said imaging device and responsive at least in part to processing at said control of captured image data by said image processor, shadows present in the field of view of said imaging device are discerned and distinguished from the other vehicle. ### Claim 49 (depends on 48) The imaging system of claim 48, wherein said imaging system determines that the object of interest present in the field of view of said imaging device comprises another vehicle responsive at least in part to processing at said control of captured image data by said image processor detecting a wheel of the other vehicle. ### Claim 50 (depends on 48) The imaging system of claim 48, wherein, responsive at least in part to processing at said control of captured image data by said image processor, a driver of the equipped vehicle is alerted to presence of the object of interest. ### Claim 51 (depends on 48) The imaging system of claim 48, wherein said control determines the object of interest present in the field of view of said imaging device at least in part responsive to determination, at least in part via processing at said control of captured image data by said image processor, of speed of the object of interest. ### Claim 52 (depends on 48) The imaging system of claim 48, wherein use of edge detection at said control comprises detection of vertical edges of objects present in the field of view of said imaging device. ### Claim 53 (depends on 52) The imaging system of claim 52, wherein the detected vertical edges are used at least in part in detection of the object of interest at said control. ### Claim 54 (depends on 53) The imaging system of claim 53, wherein said control is operable in a daytime mode and a nighttime mode. ### Claim 55 (depends on 54) The imaging system of claim 54, wherein said control switches between the daytime mode and the nighttime mode in response to an ambient light level at the equipped vehicle. ### Claim 56 (depends on 48) The imaging system of claim 48, wherein image data captured by said imaging device is provided to said control as a digital data stream. ### Claim 57 (depends on 48) The imaging system of claim 48, wherein, in detecting the object of interest present in the field of view of said imaging device, an algorithmically executed filter is utilized that ignores detected edges that are not indicative of the object of interest. ### Claim 58 (depends on 48) The imaging system of claim 48, wherein, in detecting the object of interest present in the field of view of said imaging device, edges that are indicative of a shadow cast by a tree on a road the equipped vehicle is traveling along are ignored. ### Claim 59 (depends on 48) The imaging system of claim 48, wherein captured image data is processed at said control for an adaptive cruise control system of the equipped vehicle. ### Claim 60 (depends on 48) The imaging system of claim 48, wherein said imaging system determines that the object of interest present in the field of view of said imaging device comprises another vehicle travelling in a traffic lane adjacent to a traffic lane that the equipped vehicle is travelling along. ### Claim 61 (depends on 60) The imaging system of claim 60, wherein, in determining that the object of interest present in the field of view of said imaging device comprises another vehicle, shadows cast by objects outside of the adjacent traffic lane are discerned and distinguished from objects present in the adjacent traffic lane.