# US 8,543,277 B2 · Imaging system for vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,543,277 B2 - Document ID: US8543277B2 - Status: Granted - Kind: B2 - Application number: 13/776,100 - Filed: 2013-02-25 - Published: 2013-06-27 - Granted: 2013-09-24 - Earliest claimed: 2004-12-23 - Assignees: Magna Electronics Inc. - Applicants: Donnelly Corporation - Inventors: Michael J. Higgins-Luthman - Web page: https://patentgrep.com/patent/US8543277B2 ## CPC - G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road} - B60Q 9/008: {for anti-collision purposes} - B60R 1/12: Mirror assemblies combined with other articles, e.g. clocks - B60R 1/23: with a predetermined field of view - B60W 40/04: Traffic conditions - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 20/584: {of vehicle lights or traffic lights} - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 7/181: {for receiving images from a plurality of remote sources} - H04N 23/80: Camera processing pipelines; Components thereof - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - H04N 25/76: Addressed sensors, e.g. MOS or CMOS sensors - B60R 2001/1253: {with cameras, video cameras or video screens} - B60R 2300/105: using multiple cameras - B60R 2300/30: characterised by the type of image processing - B60R 2300/402: Image calibration - B60R 2300/802: for monitoring and displaying vehicle exterior blind spot views - B60R 2300/804: for lane monitoring - B60R 2300/806: for aiding parking - B60R 2300/8066: for monitoring rearward traffic - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - B60Q: ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A vehicular imaging system includes an imaging device that captures image data and a control for processing captured image data. The control may utilize edge detection in processing captured image data. Responsive at least in part to processing of captured image data by the control, the control determines objects of interest present in the field of view of the imaging device. The objects of interest determined to be present in the field of view of the imaging device include at least one of (i) vehicles present exterior of the vehicle, (ii) headlights of vehicles present exterior of the vehicle, (iii) a bicycle that is present exterior of the vehicle and (iv) a bicycle rider that is present exterior of the vehicle. Responsive at least in part to processing of captured image data by the control, a driver of the vehicle is alerted to a hazardous condition. ## Claims ### Claim 1 An imaging system for detecting objects exterior of a vehicle, said imaging system comprising: an imaging device comprising an array of photo-sensing elements, wherein said imaging device is disposed at a vehicle equipped with said imaging system, and wherein said imaging device captures image data; wherein, with said imaging device disposed at the equipped vehicle, said imaging device has a field of view exterior of the equipped vehicle; a control for processing image data captured by said imaging device; wherein said control utilizes edge detection in processing captured image data; wherein, responsive at least in part to processing of captured image data by said control, said control determines objects of interest present in the field of view of said imaging device; wherein objects of interest determined to be present in the field of view of said imaging device comprise at least one of (i) vehicles present exterior of the equipped vehicle and (ii) headlights of vehicles present exterior of the equipped vehicle; wherein said control is operable to determine at least one of (i) a bicycle that is present exterior of the equipped vehicle and (ii) a bicycle rider that is present exterior of the equipped vehicle; wherein, responsive at least in part to processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition; and wherein said control processes a reduced data set of said captured image data to determine objects of interest present exterior of the equipped vehicle. ### Claim 2 (depends on 1) The imaging system of claim 1, wherein said control is operable to adjust processing of captured image data to accommodate misalignment of said imaging device when said imaging device is disposed at the equipped vehicle. ### Claim 3 (depends on 1) The imaging system of claim 1, wherein said control is operable to distinguish between objects present in the field of view of said imaging device and shadows of objects. ### Claim 4 (depends on 1) The imaging system of claim 1, wherein said control is operable to at least one of (a) determine that an object present in the field of view is a headlight of a vehicle forward of the equipped vehicle, (b) determine that an object present in the field of view is a headlight of a vehicle rearward of the equipped vehicle and (b) determine that an object present in the field of view is a headlight of a vehicle that is sideward of the equipped vehicle. ### Claim 5 (depends on 1) The imaging system of claim 1, wherein said imaging device captures image data for at least one of (a) a headlamp control system for detecting a headlamp of a vehicle present in the field of view of said imaging device and (b) a lane departure warning system. ### Claim 6 (depends on 1) The imaging system of claim 1, wherein said control distinguishes a headlight of a vehicle that is present exterior of the equipped vehicle in the field of view of said imaging device from other light sources present in the field of view of said imaging device. ### Claim 7 (depends on 1) The imaging system of claim 1, wherein said control is operable to switch between daytime and nighttime algorithms in response to an ambient light level at the equipped vehicle. ### Claim 8 (depends on 1) The imaging system of claim 1, wherein said control determines that a bicycle is present exterior of the equipped vehicle at least in part responsive to a determined speed of the bicycle. ### Claim 9 (depends on 1) The imaging system of claim 1, wherein said use of said edge detection by said control comprises determination of generally vertical edges of objects present in the field of view of said imaging device. ### Claim 10 (depends on 9) The imaging system of claim 9, wherein said determined generally vertical edges are used at least in part in identification of objects of interest by said control. ### Claim 11 (depends on 1) The imaging system of claim 1, wherein, in determining objects of interest present in the field of view of said imaging device, edges that are not indicative of objects of interest are substantially ignored in order to at least one of (a) reduce processing requirements and (b) reduce false signals. ### Claim 12 (depends on 1) The imaging system of claim 1, wherein said control determines that an object of interest determined to be present in the field of view of said imaging device is a wheel. ### Claim 13 (depends on 1) The imaging system of claim 1, wherein said imaging device is disposed behind the windshield of the equipped vehicle and wherein said imaging device views through the windshield and forward of the equipped vehicle, and wherein said array of photo-sensing elements comprises a CMOS array of photo-sensing elements. ### Claim 14 (depends on 1) The imaging system of claim 1, wherein said field of view of said imaging device encompasses a portion of the equipped vehicle. ### Claim 15 (depends on 14) The imaging system of claim 14, wherein said control is operable to determine that the portion of the equipped vehicle is not an object of interest. ### Claim 16 (depends on 1) The imaging system of claim 1, wherein the field of view of said imaging device encompasses (a) a rearward field of view at least partially rearward of the equipped vehicle, (b) a sideward field of view at least partially sideward of the equipped vehicle and (c) a forward field of view at least partially forward of the equipped vehicle. ### Claim 17 (depends on 1) The imaging system of claim 1, wherein said imaging device is disposed at a side of the equipped vehicle and has a field of view at least one of rearward of the equipped vehicle and sideward of the equipped vehicle. ### Claim 18 (depends on 17) The imaging system of claim 17, wherein a portion of the side of the equipped vehicle is present in the field of view of said imaging device and wherein said control determines that the portion of the equipped vehicle is not an object of interest. ### Claim 19 (depends on 1) The imaging system of claim 1, wherein said reduced data set is representative of a target zone that is encompassed by the field of view of said imaging device. ### Claim 20 (depends on 19) The imaging system of claim 19, wherein the target zone is not inclusive of a portion of the equipped vehicle. ### Claim 21 (depends on 20) The imaging system of claim 20, wherein said control adjusts the target zone responsive to a steering angle of the wheels of the equipped vehicle. ### Claim 22 (depends on 1) The imaging system of claim 1, wherein said control is operable to select said reduced image data set in response to a change in the travel speed of the equipped vehicle. ### Claim 23 (depends on 1) The imaging system of claim 1, wherein said control is operable to select said reduced image data set based at least in part on a steering angle of the equipped vehicle. ### Claim 24 (depends on 1) The imaging system of claim 1, wherein the driver of the equipped vehicle is alerted that an object of interest is determined to be present in the field of view of said imaging device. ### Claim 25 (depends on 1) The imaging system of claim 1, wherein a video display screen is located in an interior cabin of the equipped vehicle and is viewable by the driver of the equipped vehicle, and wherein images are displayed by said video display screen that are derived, at least in part, from image data captured by said imaging device. ### Claim 26 An imaging system for detecting objects exterior of a vehicle, said imaging system comprising: an imaging device comprising a CMOS array of photo-sensing elements, wherein said imaging device is disposed at a vehicle equipped with said imaging system, and wherein said imaging device captures image data; wherein, with said imaging device disposed at the equipped vehicle, said imaging device has a field of view exterior of the equipped vehicle; a control for processing image data captured by said imaging device; wherein said control utilizes edge detection in processing captured image data; wherein, responsive at least in part to processing of captured image data by said control, said control determines objects of interest present in the field of view of said imaging device; wherein objects of interest determined to be present in the field of view of said imaging device comprise at least one of (i) vehicles present exterior of the equipped vehicle and (ii) headlights of vehicles present exterior of the equipped vehicle; wherein said control is operable to determine at least one of (i) a bicycle that is present exterior of the equipped vehicle and (ii) a bicycle rider that is present exterior of the equipped vehicle; wherein, responsive at least in part to processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition; and wherein said imaging device captures image data for at least one of (a) a headlamp control system for detecting a headlamp of a vehicle present in the field of view of said imaging device and (b) a lane departure warning system. ### Claim 27 (depends on 26) The imaging system of claim 26, wherein said control is operable to distinguish between objects present in the field of view of said imaging device and shadows of objects. ### Claim 28 (depends on 26) The imaging system of claim 26, wherein said control is operable to switch between daytime and nighttime algorithms in response to an ambient light level at the equipped vehicle. ### Claim 29 (depends on 26) The imaging system of claim 26, wherein said imaging device is disposed behind the windshield of the equipped vehicle and wherein said imaging devices views through the windshield and forward of the equipped vehicle. ### Claim 30 (depends on 26) The imaging system of claim 26, wherein said field of view of said imaging device encompasses a portion of the equipped vehicle, and wherein said control is operable to determine that the portion of the equipped vehicle is not an object of interest. ### Claim 31 (depends on 26) The imaging system of claim 26, wherein said control processes a reduced data set of said captured image data to determine objects of interest, and wherein said reduced data set is representative of a target zone that is encompassed by the field of view of said imaging device. ### Claim 32 (depends on 31) The imaging system of claim 31, wherein at least one of (i) the target zone is not inclusive of a portion of the equipped vehicle, (ii) said control is operable to select said reduced image data set in response to a function of the equipped vehicle, (iii) said control is operable to select said reduced image data set in response to a change in the travel speed of the equipped vehicle and (iv) said control is operable to select said reduced image data set in response to a steering angle of the equipped vehicle. ### Claim 33 An imaging system for detecting objects exterior of a vehicle, said imaging system comprising: an imaging device comprising an array of photo-sensing elements, wherein said imaging device is disposed at a vehicle equipped with said imaging system, and wherein said imaging device captures image data; wherein, with said imaging device disposed at the equipped vehicle, said imaging device has a field of view exterior of the equipped vehicle; wherein said imaging device is disposed behind the windshield of the equipped vehicle and wherein said imaging devices views through the windshield and forward of the equipped vehicle; a control for processing image data captured by said imaging device; wherein, responsive at least in part to processing of captured image data by said control, said control determines objects of interest present in the field of view of said imaging device; wherein said control is operable to determine at least one of (i) a bicycle that is present exterior of the equipped vehicle and (ii) a bicycle rider that is present exterior of the equipped vehicle; wherein, responsive at least in part to processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition; wherein said imaging device captures image data for at least one of (a) a headlamp control system for detecting a headlamp of a vehicle present in the field of view of said imaging device and (b) a lane departure warning system; wherein said control processes a reduced data set of said captured image data to determine objects of interest; and wherein at least one of (i) said reduced data set is representative of a target zone that is encompassed by the field of view of said imaging device and (ii) said control is operable to select said reduced image data set in response to a function of the equipped vehicle. ### Claim 34 (depends on 33) The imaging system of claim 33, wherein said control is operable to select said reduced image data set in response to a function of the equipped vehicle and wherein said function comprises at least one of (i) a change in the travel speed of the equipped vehicle and (ii) a steering angle of the equipped vehicle. ### Claim 35 (depends on 33) The imaging system of claim 33, wherein said reduced data set is representative of a target zone that is encompassed by the field of view of said imaging device and that is not inclusive of a portion of the equipped vehicle. ### Claim 36 (depends on 33) The imaging system of claim 33, wherein said control is operable to distinguish between objects present in the field of view of said imaging device and shadows of objects. ### Claim 37 (depends on 33) The imaging system of claim 33, wherein said control is operable to switch between daytime and nighttime algorithms in response to an ambient light level at the equipped vehicle. ### Claim 38 (depends on 33) The imaging system of claim 33, wherein said control utilizes edge detection in processing captured image data. ### Claim 39 (depends on 33) The imaging system of claim 33, wherein said field of view of said imaging device encompasses a portion of the equipped vehicle, and wherein said control is operable to determine that the portion of the equipped vehicle is not an object of interest. ### Claim 40 (depends on 33) The imaging system of claim 33, wherein objects of interest determined to be present in the field of view of said imaging device comprise at least one of (i) vehicles present exterior of the equipped vehicle and (ii) headlights of vehicles present exterior of the equipped vehicle. ### Claim 41 An imaging system for detecting objects exterior of a vehicle, said imaging system comprising: an imaging device comprising an array of photo-sensing elements, wherein said imaging device is disposed at a vehicle equipped with said imaging system, and wherein said imaging device captures image data; wherein, with said imaging device disposed at the equipped vehicle, said imaging device has a field of view exterior of the equipped vehicle; wherein said imaging device is disposed behind the windshield of the equipped vehicle and wherein said imaging devices views through the windshield and forward of the equipped vehicle; a control for processing image data captured by said imaging device; wherein said control utilizes edge detection in processing captured image data; wherein, responsive at least in part to processing of captured image data by said control, said control determines objects of interest present in the field of view of said imaging device; wherein said control is operable to determine at least one of (i) a bicycle that is present exterior of the equipped vehicle and (ii) a bicycle rider that is present exterior of the equipped vehicle; wherein, responsive at least in part to processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition; and wherein said imaging device captures image data for at least one of (a) a headlamp control system for detecting a headlamp of a vehicle present in the field of view of said imaging device and (b) a lane departure warning system. ### Claim 42 (depends on 41) The imaging system of claim 41, wherein said control processes a reduced data set of said captured image data to determine objects of interest, and wherein said reduced data set is representative of a target zone that is encompassed by the field of view of said imaging device. ### Claim 43 (depends on 42) The imaging system of claim 42, wherein at least one of (i) the target zone is not inclusive of a portion of the equipped vehicle, (ii) said control is operable to select said reduced image data set in response to a function of the equipped vehicle, (iii) said control is operable to select said reduced image data set in response to a change in the travel speed of the equipped vehicle and (iv) said control is operable to select said reduced image data set in response to a steering angle of the equipped vehicle. ### Claim 44 (depends on 41) The imaging system of claim 41, wherein at least one of (a) said control is operable to adjust processing of captured image data to accommodate misalignment of said imaging device when said imaging device is disposed at the equipped vehicle, (b) said control is operable to distinguish between objects present in the field of view of said imaging device and shadows of objects and (c) in determining objects of interest present in the field of view of said imaging device, edges that are not indicative of objects of interest are substantially ignored. ### Claim 45 An imaging system for detecting objects exterior of a vehicle, said imaging system comprising: an imaging device comprising an array of photo-sensing elements, wherein said imaging device is disposed at a vehicle equipped with said imaging system, and wherein said imaging device captures image data; wherein, with said imaging device disposed at the equipped vehicle, said imaging device has a field of view exterior of the equipped vehicle; wherein said imaging device is disposed behind the windshield of the equipped vehicle and wherein said imaging devices views through the windshield and forward of the equipped vehicle; a control for processing image data captured by said imaging device; wherein, responsive at least in part to processing of captured image data by said control, said control determines objects of interest present in the field of view of said imaging device; wherein said control is operable to determine at least one of (i) a bicycle that is present exterior of the equipped vehicle and (ii) a bicycle rider that is present exterior of the equipped vehicle; wherein, responsive at least in part to processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition; wherein said imaging device captures image data for at least one of (a) a headlamp control system for detecting a headlamp of a vehicle present in the field of view of said imaging device and (b) a lane departure warning system; and wherein at least one of (a) said control is operable to adjust processing of captured image data to accommodate misalignment of said imaging device when said imaging device is disposed at the equipped vehicle, (b) said control is operable to distinguish between objects present in the field of view of said imaging device and shadows of objects and (c) said control processes a reduced data set of said captured image data to determine objects of interest. ### Claim 46 (depends on 45) The imaging system of claim 45, wherein said control processes a reduced data set of said captured image data to determine objects of interest and wherein said reduced data set is representative of a target zone that is encompassed by the field of view of said imaging device. ### Claim 47 (depends on 46) The imaging system of claim 46, wherein at least one of (i) the target zone is not inclusive of a portion of the equipped vehicle, (ii) said control is operable to select said reduced image data set in response to a function of the equipped vehicle, (iii) said control is operable to select said reduced image data set in response to a change in the travel speed of the equipped vehicle and (iv) said control is operable to select said reduced image data set in response to a steering angle of the equipped vehicle.