# US 11,908,119 B2 · Abnormality detection device for vehicle > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,119 B2 - Document ID: US11908119B2 - Status: Granted - Kind: B2 - Application number: 17/659,386 - Filed: 2022-04-15 - Published: 2022-07-28 - Granted: 2024-02-20 - Earliest claimed: 2019-10-16 - Assignees: Denso Corporation - Applicants: Denso Corporation - Inventors: Yutaka Aoki; Keiko Akiyama - Web page: https://patentgrep.com/patent/US11908119B2 ## CPC - G06T 7/0002: {Inspection of images, e.g. flaw detection} - G01S 7/4816: {of receivers alone} - G01S 17/931: of land vehicles - H04N 23/56: provided with illuminating means - H04N 23/71: Circuitry for evaluating the brightness variation - H04N 23/81: for suppressing or minimising disturbance in the image signal generation - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - H04N 25/61: the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" - G01S 7/497: Means for monitoring or calibrating - G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In an abnormality detection device for detecting whether there is an abnormality in either of an optical sensor and a camera that are used to detect objects around a vehicle, a correlation value calculation unit calculates a correlation value between at least one of a reflection intensity image that is detected by the optical sensor emitting light toward surroundings of the vehicle and then receiving reflected light of emitted light and a background light image that is detected by the optical sensor receiving light from the surroundings of the vehicle while the optical sensor is emitting no light, and a captured image of the surroundings from the camera. An abnormality determination unit is configured to, in response to the correlation value calculated by the correlation value calculation unit being out of a predefined range, determine that there is an abnormality in either of the optical sensor and the camera. ## Claims ### Claim 1 An abnormality detection device for detecting whether there is an abnormality in either of an optical sensor and a camera that are used to detect objects around a vehicle, the abnormality detection device comprising: a correlation value calculation unit configured to calculate a correlation value between at least one of a reflection intensity image that is detected by the optical sensor emitting light toward surroundings of the vehicle and then receiving reflected light of emitted light and a background light image that is detected by the optical sensor receiving light from the surroundings of the vehicle while the optical sensor is emitting no light, and a captured image of the surroundings of the vehicle from the camera; and an abnormality determination unit configured to, in response to the correlation value calculated by the correlation value calculation unit being out of a predefined range, determine that there is an abnormality in either of the optical sensor and the camera. ### Claim 2 (depends on 1) The abnormality detection device according to claim 1, wherein each light receiving element of a light receiving unit of the optical sensor comprises a single photon avalanche diode (SPAD). ### Claim 3 (depends on 1) The abnormality detection device according to claim 1, wherein the abnormality determination unit is configured to, in response to the correlation value being out of the predefined range, provisionally determine that there is an abnormality in either of the optical sensor and the camera, the abnormality detection device further comprises a resolution adjustment unit configured to, in response to the abnormality determination unit provisionally determining that there is an abnormality in either of the optical sensor and the camera, direct the optical sensor to increase resolution of each of the reflection intensity image and the background light image, the correlation value calculation unit is configured to calculate again the correlation value between at least one of the reflection intensity image and the background light image, whose resolution has been increased by the resolution adjustment unit, and the captured image, and the abnormality determination unit is configured to, in response to the correlation value calculated again by the correlation value calculation unit being out of the predefined range, determine that there is an abnormality in either of the optical sensor and the camera. ### Claim 4 (depends on 3) The abnormality detection device according to claim 3, wherein the resolution adjustment unit is configured to, in response to the abnormality determination unit provisionally determining that there is an abnormality in either of the optical sensor and the camera, direct the optical sensor to reduce a detection region to thereby increase the resolution. ### Claim 5 (depends on 3) The abnormality detection device according to claim 3, wherein the resolution adjustment unit is configured to, in response to the abnormality determination unit provisionally determining that there is an abnormality in either of the optical sensor and the camera, direct the optical sensor to increase the resolution of a detection region for which the correlation value is out of the predefined range, and decrease the resolution of a detection region other than the detection region for which the correlation value is out of the predefined range. ### Claim 6 (depends on 3) The abnormality detection device according to claim 3, wherein each light receiving element of a light receiving unit of the optical sensor comprises a single photon avalanche diode (SPAD), and the resolution adjustment unit is configured to, in response to the abnormality determination unit provisionally determining that there is an abnormality in either of the optical sensor and the camera, direct the optical sensor to decrease a number of SPADs per pixel to thereby the increase the resolution. ### Claim 7 (depends on 1) The abnormality detection device according to claim 1, wherein the correlation value calculation unit is configured to divide at least one of the reflection intensity image and the background light image and divide the captured image into a plurality of corresponding subregions, and calculate the correlation value for each subregion between the at least one of the reflection intensity image and the background light image and the captured image, and the abnormality determination unit is configured to, in response to the correlation value for a certain subregion calculated by the correlation value calculation unit being out of the predefined range, determine that there is an abnormality in either of the optical sensor and the camera. ### Claim 8 (depends on 1) The abnormality detection device according to claim 1, wherein the correlation value calculation unit is configured to, in response to a brightness around the vehicle being at or above a predefined value, calculate the correlation value between the background light image and the captured image, and in response to the brightness around the vehicle being below the predefined value, calculate the correlation value between the reflection intensity image and the captured image.