Verification of computer vision models

An apparatus for verifying a computer vision model, comprising:

• an input interface;
• a processor;
• a memory;
• and
• an output interface;
• wherein the input interface is configured to obtain a computer vision model, which is configured to perform a computer vision function of characterising elements of observed scenes, and to obtain a visual parameter specification including at least one visual parameter set, wherein a classification or regression performance of the computer vision model is affected by an item of visual data provided based on the at least one visual parameter set;
• wherein the input interface is configured to provide a first visual data set including a subset of items of visual data compliant with the visual parameter specification, and a corresponding subset of items of groundtruth data;
• wherein the processor is configured to perform a sensitivity analysis of a plurality of performance scores output from the computer vision model over a domain of the visual parameter specification when a plurality of items of visual data are applied to the visual data set, to generate a verification parameter specification based on the visual parameter specification and the sensitivity analysis, wherein, compared to the visual parameter specification, the verification parameter specification includes at least one fewer visual parameter sets or visual parameters, and/or wherein the verification parameter specification comprises at least one visual parameter set having a reduced range;
• and
• wherein the processor is configured to verify the computer vision model by:
• providing a second visual data set including a second subset of items of visual data compliant with the verification parameter specification, and a corresponding second subset of items of groundtruth data, wherein the second visual data set is provided using the verification parameter specification;
• and
• applying at least one verification test to the computer vision model to obtain at least one verification test result.

A non-transitory computer readable medium on which is stored a computer program for verifying a computer vision model, the computer program, when executed by a processor, causing the processor to perform the following steps:

• obtaining a computer vision model configured to perform a computer vision function of characterising elements of observed scenes;
• obtaining a visual parameter specification including at least one visual parameter set, wherein a classification or regression performance of the computer vision model is affected by an item of visual data provided based on the at least one visual parameter set;
• providing a first visual data set including a subset of items of visual data compliant with the visual parameter specification, and a corresponding subset of items of groundtruth data;
• performing a sensitivity analysis of a plurality of performance scores output from the computer vision model over a domain of the visual parameter specification when a plurality of items of visual data are applied to the visual data set;
• generating a verification parameter specification based on the visual parameter specification and the sensitivity analysis, wherein, compared to the visual parameter specification, the verification parameter specification includes at least one fewer visual parameter sets or visual parameters, and/or wherein the verification parameter specification includes at least one visual parameter set having a reduced range;
• and
• verifying the computer vision model by:
• providing a second visual data set including a second subset of items of visual data compliant with the verification parameter specification, and a corresponding second subset of items of groundtruth data, wherein the second visual data set is provided using the verification parameter specification, and
• applying at least one verification test to the computer vision model to obtain at least one verification test result.