Model comparison with unknown metric importance

A computer program product comprising one or more computer readable storage media having program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:

• generate second metric data by executing a test suite on a second version of a software model, the test suite having a number of test cases;
• generate a first version principal component matrix including first version principal components that represent first metric data by executing a dimension reduction technique on first metric data, the first metric data corresponding to a first version of the software model, the first version principal component matrix having a dimension equal to the number of test cases;
• generate a second version principal component matrix including second version principal components that represent the second metric data by executing the dimension reduction technique on the second metric data, the second version principal component matrix having a dimension equal to the number of test cases;
• produce, through further processing using principal component analysis, a variance array that shows an importance of each of a plurality of principal components of the second version principal components;
• determine at least one control limit based on at least one of the first version principal components;
• determine a mean value of one of the second version principal components;
• and
• determine whether the second version of the software model performs within the at least one control limit by comparing the mean value to the at least one control limit.

A system comprising:

• a processor, one or more computer readable storage media, and program instructions collectively stored on the one or more computer readable storage media, the program instructions executable to:
• generate second metric data by executing a test suite on a second version of a software model, the test suite having a number of test cases;
• generate a first version principal component matrix including first version principal components that represent first metric data by executing a dimension reduction technique on the first metric data, the first version principal component matrix having a dimension equal to the number of test cases, the first metric data corresponding to a first version of the software model;
• generate a second version principal component matrix including second version principal components that represent the second metric data by executing the dimension reduction technique on the second metric data, the second version principal component matrix having a dimension equal to the number of test cases;
• produce, through further processing using principal component analysis, a variance array that shows an importance of each of a plurality of principal components of the second version principal components;
• determine at least one control limit based on at least one of the first version principal components;
• and
• determine a mean value of one of the second version principal components.