Method and system to reduce false positives within an automated software-testing environment

#Exemplary embodiments of the present invention relate generally to the technical field of software testing automation and, in one exemplary embodiment, to methods and systems to reduce false positives within an automated software-testing environment using a comparator module, a filter application module, and a preview generator module.

#Websites have become increasingly complex venues for commerce (e.g., the buying and selling of goods and services over the Internet) and now include interactive functionality and sophistication that allows communication between users and a variety of databases. As such, it has become increasingly difficult to manage transitions from one software revision of a website to another software revision of the website without the use of engineers that test quality and stability of the website prior to release. Often times, testing a portion of a website affects operations within a remainder of the website because changing parameters and variables of the portion may affect the input variables of the remainder (e.g., new code can sometime break old functionality because files being changed have input and output parameters that affect the stability of the remainder). Furthermore, multiple engineers may work on the same portion of software code and a particular engineer may inadvertently modify a parameter that will affect the stability of the remainder (e.g., an engineering team may be focused on solving one problem, and may inadvertently create new problems by changing parameters that are required by another portion of software code associated with the website).

#Technology to aid testing of software revisions of websites has largely been limited to external testing tools that require an engineering team to learn a special programming language to define what aspects of the website will be tested (e.g., SILKCENTRAL™ Test Manager by Segue Software, Inc., and MERCURY WINRUNNER™ by Mercury Interactive, Inc.), and internally developed testing systems that are dependent upon manually testing all aspects of a software revision prior to release. External testing tools require that a website owner hire specialized engineers that are trained in specialized programming languages associated with the external tools. Furthermore, external tools cannot test aspects (e.g., physical locations of images, text, or input boxes on a web page) of a software revision that have not been hard coded as candidates for testing (e.g., an engineer must predetermine which aspects of a website are to be tested and hard code instructions for testing the predetermined aspects). Manual offline testing systems are error prone because engineers may not be able to test all aspects of a software revision prior to release (e.g., engineers may make a mistake, or may not have the manpower needed to test every aspect of a website).

#The present invention is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which:

#FIG. 1 is a block diagram illustrating a system having a testing device, a sniffer server, and a hosting server to test programming code associated with a website.

#FIG. 2 is a block diagram illustrating a zoom view of the testing device having a toolbar, a comparator, and a filter module, according to one exemplary embodiment.

#A method and system to reduce the number false positives within an automated software-testing environment using a comparator module, a filter application module, and a preview generator module are described. In one exemplary embodiment, a method within a testing system (e.g., a testing device) for testing web pages associated with a current version website by automatically generating filter data by comparing differences between a plurality of iterations of a test case executed on a previous version website, automatically generating error data by executing the test case on the current version website, and detecting differences between the previous version and the current version of the website by applying filtering rules within the filter data to the error data to remove false positives. In another embodiment, a merged image file may be generated by overlaying an inverted image file of a first markup language document (e.g., a web page) associated with the previous version website, and a non-inverted image file of a second markup language document associated with a current version website.

#In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of an embodiment of the present invention. It will be evident, however, to one skilled in the art that the present invention may be practiced without these specific details.

#FIG. 1 is a block diagram illustrating a system having a testing device 100, a sniffer server 106 that performs a sniffing function (e.g., captures representation data) for unencrypted web pages, and a hosting server 110 to test programming code associated with a website. The testing device 100 communicates with the sniffer server 106 and the hosting server 110 through a network 104. In one embodiment, the network 104 may be the Internet. The testing device 100 includes memory 102. The memory 102 may for example be random access memory or other volatile or nonvolatile memory. An exemplary test case may include a set of operations that define a sequence of actions a user of a website may perform (e.g., a particular test case may include a representation of actions that a typical user would take to bid on an item on a commerce website for example). Testing representations of encrypted web pages associated with a test case for a website (e.g., HTTPS XML 114A and HTTPS HTML 114B) are stored within memory 102 of the testing device 100. (e.g., the memory 102 may store testing representations of encrypted web pages when a user initiates a test case for the website 111). In contrast, testing representations of unencrypted web pages associated with the test case for the website (e.g., HTTP XML 116A and HTTP HTML 116B) are stored within a memory 108 of the sniffer server 106 so that multiple test cases can simultaneously be performed, as will later be described with reference to FIG. 3. In one embodiment, an HTTPS XML representation 114A and an HTTP XML representation 116A includes commands for regenerating the actions (e.g. movements) or operations of the user on the website 111 (e.g., movements of a user may be regenerated when the test case is automatically rerun within testing device 100). The HTTPS HTML representation 114B and HTTP HTML representation 116B may include source code of individual web pages visited within the website 111. It should be noted that the HTTPS XML 114A and HTTPS HTML 114B, as well as the HTTP XML 116A and HTTP HTML 116B may be communicated through network 104. Furthermore, in another embodiment, the sniffer server 106 and/or the testing device 100 stores all XML data required to generate the XML document for both unencrypted and/or encrypted web pages.