US 12,007,880 B2Granted

Dynamic test automation prioritization

AssigneeSap SE

InventorJohnson Wong

Life of this patent

Estimating the term…

Loading patent…

Abstract

Large and complex software projects may be distributed over multiple repositories and may use test automation of equivalent scale in continuous integration frameworks to maintain quality of the project. Such test automation often has significant hardware and time costs to run, which may mean that a failure of the software in the test automation takes longer to detect. Delay in fixing the software may increase the chance of more failures getting checked into the software repositories and perpetuating software failures. To address this issue, a ranking of historical test data is determined based on a number of failures for each test and a test configuration is determined based on the ranking such that tests that are ranked higher are performed before tests that are ranked lower. The test may be exited upon detection of failure instead of continuing.

Claims20

Dependent claims

A computer system, comprising:

one or more processors;
one or more machine-readable medium coupled to the one or more processors and storing computer program code comprising sets of instructions executable by the one or more processors to:
determine a ranking of a plurality of tests from historical test data for a software application, the historical test data including identifiers of the plurality of tests and indications of failure corresponding to the plurality of tests, the historical test data indicating a number of failures for each test of the plurality of tests previously encountered when testing the software application, the ranking of the plurality of tests based on the number of failures for each test;
determine a test configuration for testing the software application based on the ranking, the test configuration setting an order for performing the plurality of tests such that tests that are ranked higher are performed before tests that are ranked lower;
perform a set of tests on the software application based on the test configuration, the performance of the set of tests on the software application including performing the plurality of tests in the order set by the test configuration;
update the historical test data based on the performance of the set of tests to obtain updated historical test data, the updated historical test data including an identifier of any particular test that failed during the performance of the set of tests and a corresponding indication of failure;
detect that a failure is encountered in a particular test during the set of tests on the software application;
and
cease the performance of the set of tests upon the detection of the failure such that tests in the set of tests that would be performed after the particular test, according to the order set by the test configuration, are not performed.

6 more elements

One or more non-transitory computer-readable medium storing computer program code comprising sets of instructions to:

determine a ranking of a plurality of tests from historical test data for a software application, the historical test data including identifiers of the plurality of tests and indications of failure corresponding to the plurality of tests, the historical test data indicating a number of failures for each test of the plurality of tests previously encountered when testing the software application, the ranking of the plurality of tests based on the number of failures for each test;
determine a test configuration for testing the software application based on the ranking, the test configuration setting an order for performing the plurality of tests such that tests that are ranked higher are performed before tests that are ranked lower;
perform a set of tests on the software application based on the test configuration, the performance of the set of tests on the software application including performing the plurality of tests in the order set by the test configuration;
update the historical test data based on the performance of the set of tests to obtain updated historical test data, the updated historical test data including an identifier of any particular test that failed during the performance of the set of tests and a corresponding indication of failure;
detect that a failure is encountered in a particular test during the set of tests on the software application;
and
cease the performance of the set of tests upon the detection of the failure such that tests in the set of tests that would be performed after the particular test, according to the order set by the test configuration, are not performed.

4 more elements

A computer-implemented method, comprising:

determining a ranking of a plurality of tests from historical test data for a software application, the historical test data including identifiers of the plurality of tests and indications of failure corresponding to the plurality of tests, the historical test data indicating number of failures for each test of the plurality of tests previously encountered when testing the software application, the ranking of the plurality of tests based on the number of failures for each test;
determining a test configuration for testing the software application based on the ranking, the test configuration setting an order for performing the plurality of tests such that tests that are ranked higher are performed before tests that are ranked lower;
performing a set of tests on the software application based on the test configuration, the performance of the set of tests on the software application including performing the plurality of tests in the order set by the test configuration;
updating the historical test data based on the performance of the set of tests to obtain updated historical test data, the updated historical test data including an identifier of any particular test that failed during the performance of the set of tests and a corresponding indication of failure;
detecting that a failure is encountered in a particular test during the set of tests on the software application;
and
ceasing the performance of the set of tests upon the detection of the failure such that tests in the set of tests that would be performed after the particular test, according to the order set by the test configuration, are not performed.

4 more elements

Dependencies are inferred from the claim text.