# US 12,603,987 B2 · Method, system and program for data processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,603,987 B2 - Document ID: US12603987B2 - Status: Granted - Kind: B2 - Application number: 18/353,229 - Filed: 2023-07-17 - Published: 2025-01-23 - Granted: 2026-04-14 - Earliest claimed: 2023-07-17 - Assignees: Arm Limited - Applicants: Arm Limited - Inventors: Dumidu Sanjaya Talagala; Nerhun Yildiz; David Hanwell - Web page: https://patentgrep.com/patent/US12603987B2 ## CPC - H04N 17/002: {for television cameras} - G06T 7/0002: {Inspection of images, e.g. flaw detection} - G06T 7/11: Region-based segmentation - G06V 10/993: {Evaluation of the quality of the acquired pattern} - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - G06T 2207/30168: Image quality inspection - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method of data processing in a data processing system comprising a computer vision system. The method comprises obtaining image data representative of a plurality of pixels of an image, the image data comprising a plurality of pixel intensity values respectively representing said pixels. The method comprises identifying one or more compromised pixel intensity values in the plurality of pixel intensity values. The method comprises generating sensor defect state data relating to the identified compromised pixel intensity values. The method comprises performing, using the computer vision system, a feature recognition process on the image data. The method comprises performing an action based on the sensor defect state data. ## Claims ### Claim 1 A method of image processing comprising: obtaining image data representative of a plurality of pixels of an image, the image data comprising a plurality of pixel intensity values respectively representing said pixels; determining a plurality of zones of the image, each said zone comprising a respective group of said pixels, the respective groups comprising pixels from at least two rows of the image, each respective group comprising pixels from at least two columns of the image; identifying one or more compromised pixel intensity values in a plurality of given zones; generating counts of the identified compromised pixel intensity values, the counts relating to respective zones of the given zones; generating zonal sensor defect state data relating to the generated counts; and outputting the zonal sensor defect state data. ### Claim 2 (depends on 1) The method of claim 1, comprising determining whether a count of the generated counts meets a zonal defect condition, wherein the zonal sensor defect state data comprises a data element that represents whether the count meets the zonal defect condition. ### Claim 3 (depends on 1) The method of claim 1, wherein the zonal sensor defect state data comprises first zonal sensor defect state data relating to a first count of the generated counts, and second zonal sensor defect state data relating to a second count of the generated counts, and outputting the zonal sensor defect state data comprises outputting the first zonal sensor defect state data at a first time, and outputting the second zonal sensor defect state data at a second time later than the first time. ### Claim 4 (depends on 1) The method of claim 1, comprising: for each given zone, determining whether the respective count of the generated counts meets a zonal defect condition; and determining a number of the given zones in which the respective count meets the zonal defect condition, wherein the zonal sensor defect state data comprises a data element representing the number. ### Claim 5 (depends on 4) The method of claim 4, wherein the plurality of given zones forms a given macro-zone of a plurality of macro-zones of the image. ### Claim 6 (depends on 1) The method of claim 1, comprising, based on the generated counts, determining whether a collective count of the one or more compromised pixel intensity values for the image meets a sensor defect condition, wherein the zonal sensor defect state data comprises a data element representing whether the collective count meets the sensor defect condition. ### Claim 7 (depends on 6) The method of claim 6, wherein the plurality of zones comprises a further zone, and the method comprises: identifying one or more compromised pixel intensity values in the further zone; generating a further count of the identified compromised pixel intensity values in the further zone; generating further zonal sensor defect state data relating to the further count; and outputting the further zonal sensor defect state data, wherein outputting the zonal sensor defect state data is performed at a first time, and outputting the further zonal sensor defect state data is performed at a second time later than the first time. ### Claim 8 (depends on 1) The method of claim 1, wherein the image data was captured by an image sensor, the plurality of pixel intensity values represent said pixels at a first capture time, the one or more compromised pixel intensity values form a first set of compromised pixel intensity values, and the method comprises: obtaining further image data representative of a further image captured by the image sensor, the further image data comprising a plurality of further pixel intensity values respectively representing, at a second, different, capture time, at least the pixels in a particular zone, wherein the plurality of further pixel intensity values comprises a second set of one or more compromised pixel intensity values; identifying compromised pixel intensity values in the particular zone; generating a later count of the identified compromised pixel intensity values, the later count relating to the particular zone; generating further sensor defect state data relating to the later count; and comparing the zonal sensor defect state data with the further sensor defect state data, thereby monitoring a change in a defect state of the image sensor. ### Claim 9 (depends on 1) The method of claim 1, wherein the method is performed in a data processing system comprising a computer vision system, and the method comprises: performing, using the computer vision system, a feature recognition process on the image data; and performing an action based on the zonal sensor defect state data. ### Claim 10 (depends on 1) The method of claim 1, wherein the zonal sensor defect state data comprises a defect likelihood for a given said pixel intensity value. ### Claim 11 (depends on 9) The method of claim 9, wherein performing the feature recognition process comprises: determining a feature presence likelihood for a feature in the image; comparing the feature presence likelihood to a detection condition, wherein the detection condition is dependent on the zonal sensor defect state data; and determining that the feature is present if the feature presence likelihood meets the detection condition. ### Claim 12 (depends on 1) The method of claim 1, comprising: generating a count of the identified compromised pixel intensity values; and determining whether the count meets a sensor defect condition, wherein the sensor defect state data comprises a data element that represents whether the count meets the sensor defect condition. ### Claim 13 (depends on 12) The method of claim 12, wherein the image data was captured by an image sensor mounted to a vehicle, and if it is determined that the count meets the sensor defect condition, the action comprises: generating an alert that indicates that the sensor defect condition is met; and/or maneuvering the vehicle. ### Claim 14 (depends on 9) The method of claim 9, wherein the image data was captured by an image sensor mounted to a vehicle, and the action comprises maneuvering the vehicle. ### Claim 15 (depends on 9) The method of claim 9, wherein the image data was captured at a first time and the method comprises: obtaining further image data representative of a plurality of further pixels of a further image, the further image data comprising a plurality of further pixel intensity values respectively representing said further pixels, wherein the further image data was captured at a second time later than the first time; identifying one or more further compromised pixel intensity values in the plurality of further pixel intensity values; comparing pixel locations corresponding to the identified compromised pixel intensity values to pixel locations corresponding to the identified further compromised pixel intensity values; and based on said comparison, identifying a movement of at least one of the identified compromised pixel intensity values. ### Claim 16 (depends on 15) The method of claim 15, wherein: identifying the one or more compromised pixel intensity values comprises comparing at least one of the plurality of pixel intensity values to a defect condition for detecting defects; and the action comprises, in response to identifying the movement, modifying the defect condition to obtain a further defect condition for detecting defects. ### Claim 17 (depends on 15) The method of claim 15, wherein: the method comprises correcting the one or more compromised pixel intensity values using a correction method for correcting defects; and the action comprises, in response to identifying the movement, modifying the correction method to obtain a further correction method for correcting defects. ### Claim 18 A system comprising a data processor and data storage, the data storage comprising instructions which, when executed by the data processor, cause the data processor to: obtain image data representative of a plurality of pixels of an image, the image data comprising a plurality of pixel intensity values respectively representing said pixels; determine a plurality of zones of the image, each said zone comprising a respective group of said pixels, the respective groups comprising pixels from at least two rows of the image, each respective group comprising pixels from at least two columns of the image; identify one or more compromised pixel intensity values in a plurality of given zones; generate counts of the identified compromised pixel intensity values, the counts relating to respective zones of the given zones; generate zonal sensor defect state data relating to the generated counts; and output the zonal sensor defect state data. ### Claim 19 A non-transitory computer readable medium comprising instructions which, when executed by a data processor, cause the data processor to: obtain image data representative of a plurality of pixels of an image, the image data comprising a plurality of pixel intensity values respectively representing said pixels; determine a plurality of zones of the image, each said zone comprising a respective group of said pixels, the respective groups comprising pixels from at least two rows of the image, each respective group comprising pixels from at least two columns of the image; identify one or more compromised pixel intensity values in a plurality of given zones; generate counts of the identified compromised pixel intensity values, the counts relating to respective zones of the given zones; generate zonal sensor defect state data relating to the generated counts; and output the zonal sensor defect state data.