Incident tracker

A system for software testing, the system comprising:

• one or more processors;
• a memory;
• a regression identification engine configured to:
• determine a set of useable revisions relative to a current revision of source code;
• for each specific simulation of one or more simulations:
• for each specific usable revision of the useable revisions:
• execute a specific simulation of the specific useable revision;
• and
• compute a value of a metric for the specific simulation;
• compute a first average value of a set of values of the metric corresponding to a first portion of the set of usable revisions;
• compute a second average value of a set of values of the metric corresponding to a second portion of the set of usable revisions;
• and
• compute a delta value of the metric between the first average value and the second average value;
• compute a combined delta value based on the delta values computed for each specific simulation;
• compute a regression confidence interval for the current revision by computing a range for a probability distribution based on the combined delta value;
• compute a regression confidence value for the current revision by computing a probability density based on the combined delta value;
• and
• identify the current revision as a potential regression revision based on a comparison of the regression confidence interval with the regression confidence value.

A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform operations for software testing, the operations comprising:

• for each specific simulation of one or more simulations:
• for each specific usable revision of a set of useable revisions:
• computing a value of a metric for the specific simulation of the specific usable revision;
• computing a first average value of a set of values of the metric corresponding to a first portion of the set of usable revisions;
• computing a second average value of a set of values of the metric corresponding to a second portion of the set of usable revisions;
• computing a delta value of the metric between the first average value and the second average value;
• computing a combined delta value and a standard deviation value based on the delta values computed for each specific simulation;
• computing a regression confidence interval for a current revision by computing a range for a probability distribution based on the combined delta value and the standard deviation value;
• computing a regression confidence value for the current revision by computing a probability density based on the combined delta value and the standard deviation value;
• and
• determining whether the current revision is a potential regression revision based on a comparison of the regression confidence interval with the regression confidence value.