# US 11,993,274 B2 · Monitoring of on-board vehicle image capturing device functionality compliance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,993,274 B2 - Document ID: US11993274B2 - Status: Granted - Kind: B2 - Application number: 17/022,948 - Filed: 2020-09-16 - Published: 2022-03-17 - Granted: 2024-05-28 - Earliest claimed: 2020-09-16 - Assignees: Zenuity AB - Applicants: Zenuity AB - Inventors: Robert Krutsch; Valor Yaldo - Web page: https://patentgrep.com/patent/US11993274B2 ## CPC - B60W 50/0205: {Diagnosing or detecting failures; Failure detection models} - G06T 7/0002: {Inspection of images, e.g. flaw detection} - B60W 50/04: Monitoring the functioning of the control system - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06T 7/97: {Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85)} - B60W 2050/021: {Means for detecting failure or malfunction} - B60W 2050/0215: {Sensor drifts or sensor failures} - B60W 2420/403: Image sensing, e.g. optical camera - G06T 2207/10016: Video; Image sequence - G06T 2207/30168: Image quality inspection - G06T 2207/30252: Vehicle exterior; Vicinity of vehicle - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The present disclosure relates to a method performed by a luminance assessment system of a vehicle for monitoring of on-board vehicle image capturing device functionality compliance with a predeterminable requirement level. The luminance assessment system obtains at respective one or more time instants (t1, t2, t3), a first image and a second image. A first section of the first images and a second section of the second images respectively cover a primary region. A first luminance value of the first section and a second luminance value of the second section are measured. A luminance deviation is determined by comparing at least one of the first luminance values to at least one of the second luminance values. A determination is made that, when the luminance deviation exceeds a deviation threshold, the first image capturing device or the second image capturing device failed to function according to the requirement level. ## Claims ### Claim 1 A method performed by a luminance assessment system of a vehicle for monitoring of on-board vehicle image capturing device functionality compliance with a predeterminable requirement level, the vehicle comprising two or more image capturing devices configured to capture surroundings of the vehicle, a first image capturing device having a first field of view and a second image capturing device having a second field of view with a primary region thereof at least partly overlapping the first field of view, the method comprising: obtaining at respective one or more time instants (t1, t2, t3), a respective first image with support from the first image capturing device and a respective second image with support from the second image capturing device, a first section of the respective first image and a second section of the respective second image respectively covering the primary region; measuring for respective first image a respective first luminance value based on measurements of a plurality of pixels of the first section, and for respective second image a respective second luminance value based on measurements of a plurality of pixels of the second section; determining a luminance deviation exceeds a deviation threshold, the luminance deviation being based on a comparison of at least one of the respective first luminance values to at least one of the respective second luminance values; determining, with support from a positioning system, a position of the vehicle; localizing, based on the position of the vehicle, the vehicle in a digital map; determining, based on the position of the vehicle in the digital map, that there is no presence of an image luminance affecting element within an image capturing device range of the vehicle, the image luminance affecting element being a light-obstructing element; and determining, based on the luminance deviation exceeding the deviation threshold and there being no presence of the image luminance affecting element within the image capturing device range of the vehicle, that one of the first image capturing device and the second image capturing device fails to function according to the requirement level. ### Claim 2 (depends on 1) The method according to claim 1, wherein determining a luminance deviation comprises comparing an average of two or more of the respective first luminance values to an average of two or more of the respective second luminance values. ### Claim 3 (depends on 2) The method according to claim 2, wherein a third image capturing device has a third field of view with a secondary region thereof at least partly overlapping the first field of view, the method further comprising: obtaining at respective one or more time instants (t1, t2, t3), a respective third image with support from the third image capturing device, a third section of the respective third image and a first secondary section of the respective first image respectively covering the secondary region; measuring for respective third image a respective third luminance value based on measurements of a plurality of pixels of the third section, and for respective first image a respective first secondary luminance value of the first secondary section; determining a secondary luminance deviation based on comparing at least one of the respective third luminance values to at least one of the respective first secondary luminance values; and identifying at least one of: when the secondary luminance deviation exceeds a secondary deviation threshold, that it is the first image capturing device that fails to function according to the requirement level; and when the secondary luminance deviation does not exceed the secondary deviation threshold, that it is the second image capturing device that fails to function according to the requirement level. ### Claim 4 (depends on 1) The method according to claim 1, wherein a third image capturing device has a third field of view with a secondary region thereof at least partly overlapping the first field of view, the method further comprising: obtaining at respective one or more time instants (t1, t2, t3), a respective third image with support from the third image capturing device, a third section of the respective third image and a first secondary section of the respective first image respectively covering the secondary region; measuring for respective third image a respective third luminance value based on measurements of a plurality of pixels of the third section, and for respective first image a respective first secondary luminance value based on measurements of a plurality of pixels of the first secondary section; determining a secondary luminance deviation based on comparing at least one of the respective third luminance values to at least one of the respective first secondary luminance values; and identifying at least one of: when the secondary luminance deviation exceeds a secondary deviation threshold, that it is the first image capturing device that fails to function according to the requirement level; and when the secondary luminance deviation does not exceed the secondary deviation threshold, that it is the second image capturing device that fails to function according to the requirement level. ### Claim 5 (depends on 4) The method according to claim 4, wherein the determining a secondary luminance deviation comprises comparing an average of two or more of the respective third luminance values to an average of two or more of the respective first secondary luminance values. ### Claim 6 (depends on 3) The method according to claim 3, wherein the determining a secondary luminance deviation comprises comparing an average of two or more of the respective third luminance values to an average of two or more of the respective first secondary luminance values. ### Claim 7 A luminance assessment system for a vehicle for monitoring of on-board vehicle image capturing device functionality compliance with a predeterminable requirement level, the vehicle comprising two or more image capturing devices configured to capture surroundings of the vehicle, each of the two or more image capturing devices including a camera, a first image capturing device having a first field of view and a second image capturing device having a second field of view with a primary region thereof at least partly overlapping the first field of view, the luminance assessment system comprising: a processor configured to: obtain at respective one or more time instants (t1, t2, t3), a respective first image with support from the first image capturing device and a respective second image with support from the second image capturing device, a first section of the respective first image and a second section of the respective second image respectively covering the primary region; measure for respective first image a respective first luminance value based on measurements of a plurality of pixels of the first section, and for respective second image a respective second luminance value based on measurements of a plurality of pixels of the second section; determine a luminance deviation exceeds a deviation threshold, the luminance deviation being based on a comparison of at least one of the respective first luminance values to at least one of the respective second luminance values; determine, with support from a positioning system, a position of the vehicle; localize, based on the vehicle position, the vehicle in a digital map; determine, based on the vehicle position in the digital map, that there is no presence of an image luminance affecting element within an image capturing device range of the vehicle, the image luminance affecting element being a light-obstructing element; and determine, based on the luminance deviation exceeding the deviation threshold and the processor determining there being no presence of an image luminance affecting element within the image capturing device range of the vehicle, that one of the first image capturing device and the second image capturing device fails to function according to the requirement level. ### Claim 8 (depends on 7) The luminance assessment system according to claim 7, wherein the processor is configured to compare an average of two or more of the respective first luminance values to an average of two or more of the respective second luminance values. ### Claim 9 (depends on 8) The luminance assessment system according to claim 8, wherein a third image capturing device has a third field of view with a secondary region thereof at least partly overlapping the first field of view, the processor being further configured to: obtain at respective one or more time instants (t1, t2, t3), a respective third image with support from the third image capturing device, a third section of the respective third image and a first secondary section of the respective first image respectively covering the secondary region; measure for respective third image a respective third luminance value based on measurements of a plurality of pixels of the third section, and for respective first image a respective first secondary luminance value based on measurements of a plurality of pixels of the first secondary section; determine a secondary luminance deviation based on comparing at least one of the respective third luminance values to at least one of the respective first secondary luminance values; and identify at least one of: when the secondary luminance deviation exceeds a secondary deviation threshold, that it is the first image capturing device that fails to function according to the requirement level; and when the secondary luminance deviation does not exceed the secondary deviation threshold, that it is the second image capturing device that fails to function according to the requirement level. ### Claim 10 (depends on 7) The luminance assessment system according to claim 7, wherein a third image capturing device has a third field of view with a secondary region thereof at least partly overlapping the first field of view, the processor being further configured to: obtain at respective one or more time instants (t1, t2, t3), a respective third image with support from the third image capturing device, a third section of the respective third image and a first secondary section of the respective first image respectively covering the secondary region; measure for respective third image a respective third luminance value based on measurements of a plurality of pixels of the third section, and for respective first image a respective first secondary luminance value based on measurements of a plurality of pixels of the first secondary section; determine a secondary luminance deviation based on comparing at least one of the respective third luminance values to at least one of the respective first secondary luminance values; and identify at least one of: when the secondary luminance deviation exceeds a secondary deviation threshold, that it is the first image capturing device that fails to function according to the requirement level; and when the secondary luminance deviation does not exceed the secondary deviation threshold, that it is the second image capturing device that fails to function according to the requirement level. ### Claim 11 (depends on 9) The luminance assessment system according to claim 9, wherein the processor is further configured to compare an average of two or more of the respective third luminance values to an average of two or more of the respective first secondary luminance values. ### Claim 12 (depends on 10) The luminance assessment system according to claim 10, wherein the processor is further configured to compare an average of two or more of the respective third luminance values to an average of two or more of the respective first secondary luminance values. ### Claim 13 (depends on 8) The luminance assessment system according to claim 8, wherein the processor is further configured to: determine, when the luminance deviation exceeds a luminance deviation threshold and when there is determined no presence of an image luminance affecting element, that one of the first image capturing device and the second image capturing device fails to function according to the requirement level. ### Claim 14 (depends on 7) The luminance assessment system according to claim 7, wherein the processor is further configured to: determine, when the luminance deviation exceeds a luminance deviation threshold and when there is determined no presence of an image luminance affecting element, that one of the first image capturing device and the second image capturing device fails to function according to the requirement level. ### Claim 15 A vehicle comprising: a luminance assessment system, the luminance assessment system for monitoring of on-board vehicle image capturing device functionality compliance with a predeterminable requirement level, the vehicle comprising two or more image capturing devices configured to capture surroundings of the vehicle, each of the two or more image capturing devices including a camera, a first image capturing device having a first field of view and a second image capturing device having a second field of view with a primary region thereof at least partly overlapping the first field of view, the luminance assessment system comprising: a processor configured to: obtain at respective one or more time instants (t1, t2, t3), a respective first image with support from the first image capturing device and a respective second image with support from the second image capturing device, a first section of the respective first image and a second section of the respective second image respectively covering the primary region; measure for respective first image a respective first luminance value based on measurements of a plurality of pixels of the first section, and for respective second image a respective second luminance value based on measurements of a plurality of pixels of the second section; determine a luminance deviation exceeds a deviation threshold, the luminance deviation being based on a comparison of at least one of the respective first luminance values to at least one of the respective second luminance values; determine, with support from a positioning system, a position of the vehicle; and localize based on the position of the vehicle, the vehicle in view of a digital map; determine, based on the position of the vehicle, that there is no presence of an image luminance affecting element within an image capturing device range of the vehicle, the image luminance affecting element being a light-obstructing element; and determine, based on the luminance deviation exceeding the deviation threshold and there being no presence of the image luminance affecting element within an image capturing device range of the vehicle, that one of the first image capturing device and the second image capturing device fails to function according to the requirement level. ### Claim 16 A computer readable storage medium having stored thereon an executable computer program that, when executed, performs a method for a vehicle for monitoring of on-board vehicle image capturing device functionality compliance with a predeterminable requirement level, the vehicle comprising two or more image capturing devices configured to capture surroundings of the vehicle, a first image capturing device having a first field of view and a second image capturing device having a second field of view with a primary region thereof at least partly overlapping the first field of view, the method comprising: obtaining at respective one or more time instants (t1, t2, t3), a respective first image with support from the first image capturing device and a respective second image with support from the second image capturing device, a first section of the respective first image and a second section of the respective second image respectively covering the primary region; measuring for respective first image a respective first luminance value based on measurements of a plurality of pixels of the first section, and for respective second image a respective second luminance value based on measurements of a plurality of pixels of the second section; determining a luminance deviation exceeds a deviation threshold, the luminance deviation being based on a comparison of at least one of the respective first luminance values to at least one of the respective second luminance values; determining, with support from a positioning system, a position of the vehicle; localizing based on the position of the vehicle, the vehicle in view of a digital map; determining, based on the position of the vehicle in the digital map, that there is no presence of an image luminance affecting element within an image capturing device range of the vehicle, the image luminance affecting element being a light-obstructing element; and determining, based on the luminance deviation exceeding the deviation threshold and there being no presence of the image luminance affecting element within the image capturing device range of the vehicle, that one of the first image capturing device and the second image capturing device fails to function according to the requirement level.