System and method for risk-based testing

A system for risk-based testing, comprising:

• a database that stores data corresponding to a plurality of test results of test runs conducted over a certain period of time for a set of plurality of tests and data corresponding to a change set;
• and
• a processor operatively connected with the database via a communication network, wherein the processor is configured to:
• access the database to obtain the data corresponding to the plurality of test results of test runs conducted over the certain period of time for the set of plurality of tests and the data corresponding to the change set;
• generate historical data based on the data corresponding to the plurality of test results and the change set;
• assign a test failing probability value for each test among the set of plurality of tests based on analyzing the historical data;
• identify a first test whose assigned test failing probability value is below a predetermined threshold value;
• filter out the identified first test from the set of plurality of tests whose assigned test failing probability value is below the predetermined threshold value so that the identified first test is not utilized for future test runs;
• identify a second test whose assigned test failing probability value is above the predetermined threshold value;
• maintain the identified second test within the set of plurality of tests whose assigned test failing probability value is above the predetermined threshold value so that the identified second test can be utilized for the future test runs;
• identify a plurality of second tests each of whose assigned test failing probability value is above the predetermined threshold value;
• assign a priority value to each test of the plurality of second tests, wherein the priority value is proportional to corresponding failure rate of each test of the plurality of second tests within the test runs conducted over the certain period of time;
• run, for the future test runs, each test of the plurality of second tests according to an order of assigned priority value;
• project a pass/fail vector onto a normalized weight vector to only pull out weight values for test runs which have failed;
• discard or penalize for passed test cases while retaining a relationship that test runs which failed longer in the past should have less weight than those which have failed more recently;
• identify a plurality of first tests whose assigned test failing probability value is zero;
• filter out the identified plurality of first tests from the set of plurality of tests whose assigned test failing probability value is zero so that the identified plurality of first tests are not utilized for the future test runs;
• and
• run regression testing for the future test runs only for the identified plurality of second tests that have a probability of failure greater than zero and not running the identified plurality of first tests whose assigned test failing probability value is zero for the future test runs, thereby executing the regression testing in continuous delivery.

A non-transitory computer readable medium storing instructions for risk-based testing of an application, wherein when executed, the instructions cause a processor to perform the following:

• accessing a database to obtain data corresponding to a plurality of test results of test runs conducted over a certain period of time for a set of plurality of tests and data corresponding to a change set;
• generating historical data based on the data corresponding to the plurality of test results and the change set;
• assigning a test failing probability value for each test among the set of plurality of tests based on analyzing the historical data;
• identifying a first test whose assigned test failing probability value is below a predetermined threshold value;
• filtering out the identified first test from the set of plurality of tests whose assigned test failing probability value is below the predetermined threshold value so that the identified first test is not utilized for future test runs:
• identifying a second test whose assigned test failing probability value is above the predetermined threshold value;
• maintaining the identified second test within the set of plurality of tests whose assigned test failing probability value is above the predetermined threshold value so that the identified second test can be utilized for the future test runs:
• identifying a plurality of second tests each of whose assigned test failing probability value is above the predetermined threshold value;
• assigning a priority value to each test of the plurality of second tests, wherein the priority value is proportional to corresponding failure rate of each test of the plurality of second tests within the test runs conducted over the certain period of time;
• running, for the future test runs, each test of the plurality of second tests according to an order of assigned priority value;
• projecting a pass/fail vector onto a normalized weight vector to only pull out weight values for test runs which have failed;
• and
• discarding or penalizing for passed test cases while retaining a relationship that test runs which failed longer in the past should have less weight than those which have failed more recently identifying a plurality of first tests whose assigned test failing probability value is zero;
• filtering out the identified plurality of first tests from the set of plurality of tests whose assigned test failing probability value is zero so that the identified plurality of first tests are not utilized for the future test runs;
• and
• running regression testing for the future test runs only for the identified plurality of second tests that have a probability of failure greater than zero and not running the identified plurality of first tests whose assigned test failing probability value is zero for the future test runs, thereby executing the regression testing in continuous delivery.