Method and system for generating a computer program test suite using dynamic symbolic execution of JAVA programs

A method using a computer for generating a test suite for a computer program written in the JAVA programming language, the computer program comprising program statements and program variables and being represented as JAVA bytecodes after being compiled by a JAVA compiler, the computer program including at least one input statement having one or more input variables, grouped into code blocks, the test suite comprising sets of inputs, each of the sets of inputs corresponding to each of the input statements, comprising the steps of:

• reading the program statements corresponding to a candidate code block from the JAVA bytecodes;
• representing each of the input variables for each input statement and each of the program variables in symbolic form as a symbolic memory value and transforming each program statement dependent on such an input variable into a symbolic expression;
• creating a trial set of inputs for each of the input statements by finding a solution to the symbolic expression comprising actual input values corresponding to each symbolic memory value using dynamic symbolic execution of the JAVA bytecodes;
• interpreting the JAVA bytecodes using the trial set of inputs and analyzing results obtained for coverage of the candidate code block;
• and
• storing the trial set of inputs into the test suite if coverage of the candidate code block was obtained.

A system for generating a test suite for a computer program written in the JAVA programming language, the computer program comprising program statements and program variables and represented as JAVA bytecodes after being compiled by a JAVA compiler, the computer program including at least one input statement having one or more input variables, grouped into code blocks and stored in a program database, the test suite comprising sets of inputs, each of the sets of inputs corresponding to each of the input statements, comprising:

• means for reading the program statements corresponding to a candidate code block from the JAVA bytecodes;
• means for representing each of the input variables for each input statement and each of the program variables in symbolic form as a symbolic memory value and means for transforming each program statement dependent on such an input variable into a symbolic expression;
• means for creating a trial set of inputs for each of the input statements by finding a solution to the symbolic expression comprising actual input values corresponding to each symbolic memory value using dynamic symbolic execution of the JAVA bytecodes;
• means for interpreting the JAVA bytecodes using the trial set of inputs and means for analyzing results obtained for coverage of the candidate code block;
• and
• means for storing the trial set of inputs into the test suite if coverage of the candidate code block was obtained.

A method of operating a computer system for performing dynamic symbolic execution by a symbolic virtual machine of a computer program written in the JAVA programming language and represented as JAVA bytecodes after being compiled by a JAVA compiler, the computer system having a memory, the method comprising the steps of:

• reading a JAVA bytecode to be executed;
• writing memory modifications into selected memory areas of the symbolic virtual machine when the JAVA bytecode being executed modifies a location in computer system memory;
• resolving any undefined external references included in the JAVA bytecode;
• and
• repeating the reading, writing and resolving steps for all JAVA bytecodes in the computer program.

A computer-implemented method of detecting runtime errors in a computer program written in the JAVA programming language, the computer program being represented by JAVA bytecodes after being compiled by a JAVA compiler, the computer-implemented method comprising the steps of:

• reading the JAVA bytecodes;
• obtaining at least one input value for the computer program;
• symbolically executing an instruction of the computer program represented in the JAVA bytecodes;
• examining all symbolic expressions on which the instruction depends based on the symbolic execution;
• and
• storing the at least one input value in a test database and marking the at least one input value as an input that generates a runtime error when a solution to the symbolic expressions generates an error condition.

A computer-implemented method of generating a test suite for a module of an incomplete application program written in the JAVA programming language having undefined external references or a missing main method, the application program being represented by JAVA bytecodes after being compiled by a JAVA compiler, the computer-implemented method comprising the steps of:

• reading the JAVA bytecodes for the module;
• determining a sequence of JAVA methods to be called in the module and arguments to be passed to the JAVA methods;
• performing symbolic interpretation of the sequence of JAVA methods of the JAVA bytecodes by calculating symbolic expressions for the JAVA methods and using the symbolic expressions to determine input values causing a predetermined testing criteria to be satisfied;
• storing the input values in a test suite database when solutions for the predetermined testing criteria based on the symbolic expressions are found for the input values;
• and
• repeating the determining and performing steps when solutions are not found for the predetermined testing criteria based on the symbolic expressions.

A computer-implemented method of inserting instrumentation information into JAVA bytecodes produced by compiling a computer program written in the JAVA programming language, comprising the steps of:

• reading the JAVA bytecodes and the computer program;
• creating a parse tree for the computer program and analyzing the parse tree to identify points in the computer program where instrumentation can be inserted;
• analyzing the JAVA bytecodes and instrumenting the JAVA bytecodes at the identified points;
• and
• writing a new set of JAVA bytecodes including the instrumentation information.