Method and apparatus for environment-variable driven software testing

A method for performing a test on software in a specified environment in a computer system, the computer system including a storage device containing a test base, the test base including a plurality of directories connected to form a hierarchical directory structure, wherein a first directory of said plurality of directories corresponds to said test, wherein a plurality of environment configuration files are stored in said plurality of directories, the method comprising the steps of:

• determining which of said plurality of environment configuration files are applicable environment configuration files based on where said plurality of environment configuration files are located in said hierarchical directory structure relative to said first directory;
• creating said specified environment based on said applicable environment configuration files;
• and
• testing said software in said specified environment.

A method for performing a test on software using a test base on a computer-readable storage device, the method comprising the computer-implemented steps of:

• creating a plurality of directories on said computer-readable storage device, wherein said plurality of directories are connected to form a hierarchical directory structure, wherein a first directory of said plurality of directories corresponds to a first test and a second directory of said plurality of directories corresponds to a second test;
• storing a plurality of environment configuration files in said plurality of directories, wherein said environment configuration files include a first set of environment configuration files that apply to said first test but not said second test, a second set of environment configuration files that apply to said second test but not said first test, and a third set of environment configuration files that apply to both said first test and said second test;
• and
• performing a test on software using said plurality of directories and said plurality of environment configuration files, said performing step further including a step of determining which of said plurality of environment configuration files are applicable to said test based on a location of said environment configuration files in said plurality of directories.

A method for performing a second series of tests on a plurality of software programs based on results of a first series of tests performed on said plurality of software programs, the method comprising the computer implemented steps of:

• performing said first series of tests, wherein each test of said first series of tests is performed on a specified software/environment combination, wherein each software/environment combination includes a specified software program of said plurality of software programs and a specified environment of a plurality of predefined environments;
• storing on a storage device result data indicating results of said first series of tests, said result data including data indicating whether a most recent performance of each test of said first series of tests was successful;
• reading said result data from said storage device;
• generating a test list based on said result data, wherein said test list includes an ordered list of entries, wherein each entry of said test list represents a test performed in said first series of tests, wherein each entry in said list of entries is ordered based on the outcome of the test represented by said entry during said first series of tests;
• and
• performing said second series of tests based on said test list, wherein the tests represented in said test list are performed in the order of their corresponding entries in the test list.

A method for selecting a subset of tests to run on software programs, wherein said subset of tests are a subset of a plurality of tests, the method comprising the steps of:

• A) establishing a mapping between said plurality of tests and a plurality of directories within a hierarchical directory structure on a storage device, wherein each test of said plurality of tests is mapped to at least one of said plurality of directories;
• B) for each given directory within said hierarchical directory that is mapped to one or more of said plurality of tests, performing the steps of:
• B1) generating attribute data that specifies attributes associated with said one or more of said plurality of tests;
• B2) storing said attribute data in said given directory;
• C) receiving input indicating a selected directory of said plurality of directories and a selected attribute;
• D) determining whether any tests mapped to said selected directory are associated with said selected attribute;
• E) including any tests mapped to said selected directory that are associated with said selected attribute in said subset of tests to run;
• F) determining whether any tests mapped to said child directories are associated with said selected attribute;
• and
• G) including any tests mapped to said child directories that are associated with said selected attribute in said subset of tests to run.