US 5,513,315 A · System and method for automatic testing of computer software

Abstract

A system and method for automatically testing software using a deterministic acceptance test and random command sequence selections to more rapidly uncover errors in computer software. A results analyzer checks test parameters following the execution of each of a series of predetermined test commands and a series of random test commands to determine if the commands were properly executed. The test command sequences and test results determined by the results analyzer are stored in a log file which may be examined by the tester. The randomly selected test command sequence is stored in a tracker log file. The tester may reexecute the tracker file in its entirety or any portion selected by the tester as a means of determining the cause of an error in the software. The system also provides for error recovery. When an error is detected, the system restarts the test of the computer software and continues maintaining the log file and the tracker file. The system also uses the probabilities for various responses that an end-user may make in response to a particular screen display or program state. The system may select random responses corresponding to the probabilities. These probabilities are calculated for each individual screen display or software state. Because the present invention executes a random selection of command sequences, a particular applications program may be tested on multiple machines, thereby decreasing the overall time required for acceptance testing.

Claims66

Dependent claims

A system for automatically testing computer software and detecting errors in the computer software, the system comprising:

a preliminary test file containing a series of predetermined commands for the computer software to test initial parameters of the computer software;
an operational profile containing a list of end-user choices in response to a particular program state of the computer software, said list being constructed according to probabilities of end-user choices in response to said particular program state;
a random test selector to randomly select a series of commands from said operational profile for the computer software to execute;
a tracker log file to record each of said random series of commands selected by said random test selector;
means for selecting a portion of said tracker log file;
a command simulator simulating on the computer software the selection of said predetermined commands, said random series of commands, and at least said selected portion of said tracker log;
a results analyzer analyzing at least one test parameter of the computer software after the execution by the computer software of each of said predetermined commands, said random series of commands, and said selected portion of said tracker log file to determine if each of said predetermined commands, said random series of commands, and said selected portion of said tracker log file was properly executed by the computer software, said results analyzer generating a test result following the execution of each of said predetermined commands, said random series of commands, and said selected portion of said tracker log file;
a test log file recording said test results for each of said predetermined commands, said random series of commands, and said selected portion of said tracker log file;
and
an error recovery unit automatically restarting the computer software to continue testing the computer software if one of said random series of commands was not properly executed by the computer software.

7 more elements

A system for automatically testing computer software and detecting errors in the computer software, the system comprising:

a preliminary test file containing a series of predetermined commands for the computer software to test initial parameters of the computer software;
an operational profile containing a list of end-user choices in response to a particular program state of the computer software, said list being constructed according to probabilities of end-user choices in response to said particular program state;
a random test selector to randomly select a series of commands from said operational profile for the computer software to execute;
a tracker log file to record each of said random series of commands selected by said random test selector;
a command simulator simulating on the computer software the selection of said predetermined commands, said random series of commands, and at least a portion of said tracker log;
and
a results analyzer analyzing at least one test parameter of the computer software after the execution by the computer software of each of said predetermined commands, said random series of commands, and said portion of said tracker log file to determine if each of said predetermined commands, said random series of commands, and said portion of said tracker log file was properly executed by the computer software, said results analyzer generating a test result following the execution of each of said predetermined commands, said random series of commands, and said portion of said tracker log file.

4 more elements

A system for automatically testing computer software and detecting errors in the computer software, the system comprising:

an operational profile containing a list of end-user choices in response to a particular program state of the computer software, said list being constructed according to probabilities of end-user choices in response to said particular program state;
a random test selector to randomly select a series of commands from said operational profile for the computer software to execute;
a tracker log file to record each of said random series of commands selected by said random test selector;
means for selecting a portion of said tracker log file;
a command simulator simulating on the computer software the selection of said random series of commands, and at least said selected portion of said tracker log;
a results analyzer analyzing at least one test parameter of the computer software after the execution by the computer software of each of said random series of commands, and said portion of said tracker log file to determine if each of said random series of commands and said selected portion of said tracker log file was properly executed by the computer software, said results analyzer generating a test result following the execution of each of said random series of commands and said selected portion of said tracker log file;
a test log file recording said test results for each of said random series of commands and said selected portion of said tracker log file;
and
an error recovery unit automatically restarting the computer software to continue testing the computer software if one of said random series of commands was not properly executed by the computer software.

6 more elements

A system for automatically testing computer software and detecting errors in the computer software, the system comprising:

a random test selector to randomly select a series of commands for the computer software to execute;
a tracker log file to record each of said random series of commands selected by said random test selector;
means for selecting a portion of said tracker log file;
a command simulator simulating on the computer software the selection of said random series of commands, and at least said selected portion of said tracker log;
a results analyzer analyzing at least one test parameter of the computer software after the execution by the computer software of each of said random series of commands, and said portion of said tracker log file to determine if each of said random series of commands and said selected portion of said tracker log file was properly executed by the computer software, said results analyzer generating a test result following the execution of each of said random series of commands and said selected portion of said tracker log file;
a test log file recording said test results for each of said random series of commands and said selected portion of said tracker log file;
and
an error recovery unit automatically restarting the computer software to continue testing the computer software if one of said random series of commands was not properly executed by the computer software.

5 more elements

A method for automatically testing computer software and detecting errors in the computer software, the method comprising the steps of:

(a) testing initial parameters of the computer software by having the computer software execute a series of predetermined commands for the computer software;
(b) randomly selecting a series of commands from an operational profile containing a list of end-user choices in response to a particular program state of the computer software;
(c) recording each of said randomly selected commands in a tracker log file;
(d) executing on the computer software said predetermined commands and said randomly selected commands;
(e) analyzing the computer software after the execution of each of said predetermined commands and said randomly selected commands to determine if each of said predetermined commands and said randomly selected commands was properly executed by the computer software;
(f) generating a test result following the execution of each of said predetermined commands and said randomly selected commands;
(g) recording said test results in a test log file for each of said predetermined commands and said randomly selected commands;
and
(h) automatically restarting the computer software to continue testing the computer software if one of said randomly selected commands was not properly executed by the computer software.

6 more elements

A method for automatically testing computer software and detecting errors in the computer software, the method comprising the steps of:

(a) randomly selecting a series of commands from an operational profile containing a list of end-user choices in response to a particular program state of the computer software;
(b) recording each of said randomly selected commands in a tracker log file;
(c) executing on the computer software said randomly selected commands;
(d) analyzing the computer software after the execution of each of said randomly selected commands to determine if each of said randomly selected commands was properly executed by the computer software;
(e) generating a test result following the execution of each of said randomly selected commands;
(f) recording said test results in a test log file for each of said randomly selected commands;
and
(g) automatically restarting the computer software to continue testing the computer software if one of said randomly selected commands was not properly executed by the computer software.

5 more elements

Dependencies are inferred from the claim text.

System and method for automatic testing of computer software

Life of this patent

System and method for automatic testing of computer software

Life of this patent

Abstract

Claims66