# US 2021/0241008 A1 · Vehicle vision system with object detection failsafe > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241008 A1 - Document ID: US20210241008A1 - Status: Published application - Kind: A1 - Application number: 17/249,376 - Filed: 2021-03-01 - Published: 2021-08-05 - Earliest claimed: 2017-01-23 - Assignees: Magna Electronics Inc. - Applicants: Magna Electronics Inc. - Inventors: Horst D. Diessner; Richard C. Bozich; Aleksandar Stefanovic; Anant Kumar Lall; Nikhil Gupta - Web page: https://patentgrep.com/patent/US20210241008A1 ## CPC - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08) - G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes} - G08G 1/167: {Driving aids for lane monitoring, lane changing, e.g. blind spot detection} - H04N 5/14: Picture signal circuitry for video frequency region (cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00) - B60R 2300/402: Image calibration - G06T 2207/30244: Camera pose - G06T 2207/30261: Obstacle - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93}) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method for determining a safe state for a vehicle includes disposing a camera at a vehicle and disposing an electronic control unit (ECU) at the vehicle. Image data is captured via the camera and provided to the ECU. An image processor of the ECU processes captured image data. A condition is determined via processing at the image processor of the ECU captured image data. The condition comprises a shadow present in the field of view of the camera within ten frames of captured image data or a damaged condition of the imager within two minutes of operation of the camera. The condition is indicative of a condition where processing of captured image data degrades in performance. The ECU determines a safe state for the vehicle responsive to determining the condition. ## Claims ### Claim 1 A method for determining a safe state for a vehicle, the method comprising: disposing a camera at a vehicle such that the camera has a field of view exterior of the vehicle, the camera comprising a lens and an imager; disposing an electronic control unit (ECU) at the vehicle, wherein the ECU comprises an image processor operable to process image data captured by the camera; capturing image data via the camera and providing captured image data to the ECU; processing at the image processor of the ECU image data captured by the camera and provided to the ECU; determining, via processing at the image processor of the ECU image data captured by the camera and provided to the ECU, a shadow present in the field of view of the camera within ten frames of captured image data, wherein the determined shadow present in the field of view of the camera within ten frames of captured image data is indicative of a condition where processing of captured image data degrades in performance; and determining, via the ECU, and responsive to determining the shadow present in the field of view of the camera within ten frames of captured image data, a safe state for the vehicle. ### Claim 2 (depends on 1) The method of claim 1, wherein the camera comprises a forward-viewing camera that views forward of the vehicle. ### Claim 3 (depends on 2) The method of claim 2, wherein the camera is disposed at a windshield of the vehicle and views through the windshield. ### Claim 4 (depends on 2) The method of claim 2, wherein the camera is disposed at a front grille of the vehicle. ### Claim 5 (depends on 1) The method of claim 1, wherein the camera comprises a rearward-viewing camera that views rearward of the vehicle. ### Claim 6 (depends on 1) The method of claim 1, wherein the camera comprises a sideward-viewing camera that views sideward of the vehicle. ### Claim 7 (depends on 1) The method of claim 1, further comprising determining, responsive to processing at the ECU of image data captured by the camera and provided to the ECU, if the lens of the camera is at least partially blocked. ### Claim 8 (depends on 1) The method of claim 1, further comprising determining, responsive to processing at the ECU of image data captured by the camera and provided to the ECU, if the camera is out of focus. ### Claim 9 (depends on 1) The method of claim 1, further comprising determining, responsive to processing at the ECU of image data captured by the camera and provided to the ECU, if the camera is out of calibration. ### Claim 10 (depends on 1) The method of claim 1, wherein the determined safe state accommodates for variations in vehicle roll angles and vehicle pitch angles. ### Claim 11 (depends on 1) The method of claim 1, wherein the determined safe state accommodates for a blockage condition at the lens of the camera. ### Claim 12 (depends on 1) The method of claim 1, wherein the imager of the camera comprises a CMOS imaging array having at least one million photosensor elements arranged in a plurality of rows and columns. ### Claim 13 A method for determining a safe state for a vehicle, the method comprising: disposing a camera at a vehicle such that the camera has a field of view exterior of the vehicle, the camera comprising a lens and an imager; disposing an electronic control unit (ECU) at the vehicle, wherein the ECU comprises an image processor operable to process image data captured by the camera; capturing image data via the camera and providing captured image data to the ECU; processing at the image processor of the ECU image data captured by the camera and provided to the ECU; determining, via processing at the image processor of the ECU image data captured by the camera and provided to the ECU, a damaged condition of the imager within two minutes of operation of the camera, wherein the determined damaged condition of the imager within two minutes of operation of the camera is indicative of a condition where processing of captured image data degrades in performance; and determining, via the ECU, and responsive to determining the damaged condition of the imager within two minutes of operation of the camera, a safe state for the vehicle. ### Claim 14 (depends on 13) The method of claim 13, wherein the camera comprises a forward-viewing camera that views forward of the vehicle. ### Claim 15 (depends on 14) The method of claim 14, wherein the camera is disposed at a windshield of the vehicle and views through the windshield. ### Claim 16 (depends on 14) The method of claim 14, wherein the camera is disposed at a front grille of the vehicle. ### Claim 17 (depends on 13) The method of claim 13, wherein the camera comprises a rearward-viewing camera that views rearward of the vehicle. ### Claim 18 (depends on 13) The method of claim 13, wherein the camera comprises a sideward-viewing camera that views sideward of the vehicle. ### Claim 19 (depends on 13) The method of claim 13, further comprising determining, responsive to processing at the ECU of image data captured by the camera and provided to the ECU, if the lens of the camera is at least partially blocked. ### Claim 20 (depends on 13) The method of claim 13, further comprising determining, responsive to processing at the ECU of image data captured by the camera and provided to the ECU, if the camera is out of focus. ### Claim 21 (depends on 13) The method of claim 13, further comprising determining, responsive to processing at the ECU of image data captured by the camera and provided to the ECU, if the camera is out of calibration. ### Claim 22 (depends on 13) The method of claim 13, wherein the determined safe state accommodates for variations in vehicle roll angles and vehicle pitch angles. ### Claim 23 (depends on 13) The method of claim 13, wherein the determined safe state accommodates for a blockage condition at the lens of the camera. ### Claim 24 (depends on 13) The method of claim 13, wherein the imager of the camera comprises a CMOS imaging array having at least one million photosensor elements arranged in a plurality of rows and columns. ### Claim 25 A method for determining a safe state for a vehicle, the method comprising: disposing a camera at a vehicle such that the camera has a field of view exterior of the vehicle, the camera comprising a lens and an imager, wherein the imager of the camera comprises a CMOS imaging array having at least one million photosensor elements arranged in a plurality of rows and columns; disposing an electronic control unit (ECU) at the vehicle, wherein the ECU comprises an image processor operable to process image data captured by the camera; capturing image data via the camera and providing captured image data to the ECU; processing at the image processor of the ECU image data captured by the camera and provided to the ECU; determining, via processing at the image processor of the ECU image data captured by the camera and provided to the ECU, (i) a shadow present in the field of view of the camera within ten frames of captured image data or (ii) a damaged condition of the imager within two minutes of operation of the camera, wherein the determined shadow or damaged condition of the imager is indicative of a condition where processing of captured image data degrades in performance; determining, via the ECU, and responsive to determining (i) the shadow present in the field of view of the camera within ten frames of captured image data or (ii) a damaged condition of the imager within two minutes of operation of the camera, a safe state for the vehicle; and determining, responsive to processing at the ECU of image data captured by the camera and provided to the ECU, if (i) the lens of the camera is at least partially blocked, (ii) the camera is out of focus or (iii) the camera is out of calibration. ### Claim 26 (depends on 25) The method of claim 25, wherein the camera comprises a forward-viewing camera that views forward of the vehicle. ### Claim 27 (depends on 26) The method of claim 26, wherein the camera is disposed at a windshield of the vehicle and views through the windshield. ### Claim 28 (depends on 26) The method of claim 26, wherein the camera is disposed at a front grille of the vehicle. ### Claim 29 (depends on 25) The method of claim 25, wherein the camera comprises a rearward-viewing camera that views rearward of the vehicle. ### Claim 30 (depends on 25) The method of claim 25, wherein the camera comprises a sideward-viewing camera that views sideward of the vehicle. ### Claim 31 (depends on 25) The method of claim 25, wherein the determined safe state accommodates for variations in vehicle roll angles and vehicle pitch angles. ### Claim 32 (depends on 25) The method of claim 25, wherein the determined safe state accommodates for a blockage condition at the lens of the camera.