Adaptive regression test selection within testing environments

A method comprising:

• determining, with a computing device, those functions of source code whose lines of code have changed between different versions of the software program;
• determining, with the computing device, those of the functions of the source code whose lines of code have not changed but have been impacted by the changes based on dependency data and the determined changed functions, wherein the dependency data defines dependencies between the functions of the source code;
• selecting, with the computing device, a subset of a full set of test cases that uniquely exercises every one of the changed and impacted functions based on coverage data, wherein the coverage data defines at least one function-level block combination for each test case of the full set of test cases, wherein each of the function-level block combinations comprises data that associates a different one of the test cases with one or more blocks of a particular one of the functions of the source code, wherein each of the blocks comprises a contiguous block of the source code having a single point of entry and a single point of exit;
• and
• executing, with the computing device, the subset of the full set of test cases to ensure a given level of safety with respect to a percentage of functional errors that reside within a software program compiled from the source code, wherein the given level of safety requires the computing device to select the subset of test cases from the full set of test cases such that execution of the subset of test cases reveals at least the percentage of functional errors.

A testing server comprising:

• a control unit that includes one or more processors that execute an intelligent test selection module that determines those functions of source code whose lines of code have changed between different versions of the software program, determines those of the functions of the source code whose lines of code have not changed but have been impacted by the changes based on dependency data and the determined changed functions, wherein the dependency data defines dependencies between the functions of the source code, selects one or more test cases from a full set of test cases for each one of the changed and impacted functions in accordance with an adaptive test selection algorithm, wherein the adaptive test selection algorithm adaptively applies one or more test selection algorithms in order of decreasing safety assurances and increasing precision provided by each of the test selection algorithms until one of the test selection algorithms returns a number of test cases less than a maximum test case limit and aggregates the one or more test cases selected for each one of the changed and impacted functions to generate a final test set that includes a subset of test cases from the full set of test cases,
• wherein the one or more processors also execute a regression harness that executes the final test set to ensure a given level of safety with respect to a percentage of functional errors that reside within a software program compiled from the source code, wherein the given level of safety requires the computing device to select the subset of test cases from the full set of test cases such that execution of the subset of test cases reveals at least the percentage of functional errors.

A testing system comprising:

• a test bench comprising a plurality of network devices;
• a test selection database that stores dependency data and code change data;
• a test case database that stores a full set of test cases;
• a testing server communicatively coupled to the test bench, the test selection database and the test case database, wherein the testing server includes:
• a control unit that comprises one or more processors that execute an intelligent test selection module that determines those functions of source code whose lines of code have changed between different versions of the software program based on the code change data, determines those of the functions of the source code whose lines of code have not changed but have been impacted by the changes based on the dependency data and the determined changed functions, wherein the dependency data defines dependencies between the functions of the source code, selects one or more test cases from the full set of test cases for each one of the changed and impacted functions in accordance with an adaptive test selection algorithm, wherein the adaptive test selection algorithm adaptively applies one or more test selection algorithms in order of decreasing safety assurances and increasing precision provided by each of the test selection algorithms until one of the test selection algorithms returns a number of test cases less than a maximum test case limit and aggregates the one or more test cases selected for each one of the changed and impacted functions to generate a final test set that includes a subset of test cases from the full set of test cases, and
• wherein the one or more processors also execute a regression harness that executes the final test set within the plurality of network devices of the test bench to ensure a given level of safety with respect to a percentage of functional errors that reside within a software program compiled from the source code, wherein the given level of safety requires the computing device to select the subset of test cases from the full set of test cases such that execution of the subset of test cases reveals at least the percentage of functional errors.

A non-transitory computer-readable medium comprising instructions for causing a programmable processor to:

• determine, with a computing device, those functions of source code whose lines of code have changed between different versions of the software program;
• determine, with the computing device, those of the functions of the source code whose lines of code have not changed but have been impacted by the changes based on dependency data and the determined changed functions, wherein the dependency data defines dependencies between the functions of the source code;
• select, with the computing device, one or more test cases from a full set of test cases for each one of the changed and impacted functions in accordance with an adaptive test selection algorithm, wherein the adaptive test selection algorithm adaptively applies one or more test selection algorithms in order of decreasing safety assurances and increasing precision provided by each of the test selection algorithms until one of the test selection algorithms returns a number of test cases less than a maximum test case limit;
• aggregate, with the computing device, the one or more test cases selected for each one of the changed and impacted functions to generate a final test set that includes a subset of test cases from the full set of test cases;
• and
• execute, with the computing device, the final test set to ensure a given level of safety with respect to a percentage of functional errors that reside within a software program compiled from the source code, wherein the given level of safety requires the computing device to select the subset of test cases from the full set of test cases such that execution of the subset of test cases reveals at least the percentage of functional errors.