# US 10,482,339 B2 · Quantifying computer vision algorithm performance in the presence of system uncertainty > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,482,339 B2 - Document ID: US10482339B2 - Status: Granted - Kind: B2 - Application number: 15/834,492 - Filed: 2017-12-07 - Published: 2018-06-14 - Granted: 2019-11-19 - Earliest claimed: 2016-12-09 - Assignees: The United States of America as represented by the Secretary of the Air Force - Applicants: Government of the United States as represented by the Secretary of the Air Force - Inventors: Mohammad I Vakil; John A Malas - Web page: https://patentgrep.com/patent/US10482339B2 ## CPC - G06V 10/776: Validation; Performance evaluation - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06T 5/70: Denoising; Smoothing - G06T 7/0004: {Industrial image inspection} - G06V 10/30: Noise filtering - G06T 2207/10048: Infrared image - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A method of quantifying computer vision algorithm performance. The method includes receiving a first image and a second image from an imaging system. Each of the first and second images is characterized by an image intensity value. Iteratively, an evaluation value of a noise profile is applies to the first and second images to form respective first and second composite images, and algorithm performance using the first and second composite images is measured. The measured performances are compared and an operational range of the algorithm determined. The noise profile includes at least one source of noise inherent to the imaging system. ## Claims ### Claim 1 A method of determining an operational range for a computer vision algorithm, the method comprising: receiving a first image and a second image from an imaging system, each of the first and second images characterized by an image intensity value; iteratively, applying an evaluation value of a noise profile to the first and second images to yield first and second composite images, respectively, and measuring performance of the computer vision algorithm on the first and second composite images; comparing the measure of performances; and determining an operational range of the computer vision algorithm from the comparison, wherein the noise profile includes at least one source of noise inherent to the imaging system. ### Claim 2 (depends on 1) The method of claim 1, wherein the first and second images are of a spectra type selected from the group consisting of visible band, short-wave infrared (SWIR) band, mid-wave infrared (MWIR) band, and long-wave infrared (LWIR) band. ### Claim 3 (depends on 1) The method of claim 1, wherein the at least one source of noise is selected from the group consisting of sensor noise sources, quantization noise sources, and impulse noise sources. ### Claim 4 (depends on 1) The method of claim 1, wherein applying the evaluation value of the noise profile further comprises: generating first and second noise masks for the first and second images, respectively. ### Claim 5 (depends on 4) The method of claim 4, further comprising: scaling the first and second noise masks to maximum image intensity values of the first and second images, respectively. ### Claim 6 (depends on 1) The method of claim 1, wherein the computer vision algorithm is selected from the group consisting of pattern recognition, image registration, change detection, and rotational transform. ### Claim 7 (depends on 6) The method of claim 6, wherein the computer vision algorithm is a rotational transform and the measure of performance is an angle of rotation. ### Claim 8 (depends on 1) The method of claim 1, wherein comparing the measure of performances includes selecting a minimum value, a maximum value, an average value, a standard deviation, and a variance. ### Claim 9 (depends on 1) The method of claim 1 wherein the evaluation value is within an evaluation range. ### Claim 10 (depends on 1) A system for determining an operational range of a computer vision algorithm of an imaging system, the system comprising: a processing module configured to perform the method of claim 1. ### Claim 11 (depends on 10) The system of claim 10, wherein the processing module further comprises: a receiving module configured to receive the first and second images. ### Claim 12 (depends on 10) The system of claim 10, wherein the computer vision algorithm is selected from the group consisting of pattern recognition, image registration, change detection, and rotational transform. ### Claim 13 (depends on 10) The system of claim 10, wherein the measure of performance is selected from the group consisting of a matched filter, a normalized cross correlation, and an information theoretic measure. ### Claim 14 (depends on 1) An imaging system characterized by one or more sources of noise and having a computer vision algorithm, the system comprising: an imaging sensing module for capturing a target image, the target image being characterized by the image intensity value; and a processor configured to derive a first image and a second image from the received target image and to perform the method of claim 1 thereon. ### Claim 15 (depends on 14) The imaging system of claim 14, wherein the imaging system further comprises: an image conditioning module configured to receive the target image and adjust the image intensity value thereof. ### Claim 16 (depends on 15) The imaging system of claim 15, wherein the image conditioning module comprises at least one of a filter, an amplifier, and a signal processing routine. ### Claim 17 (depends on 15) The imaging system of claim 15, wherein the imaging system is associated with at least one of a sensor noise source, a quantization noise source, and an impulse noise source.