# US 9,519,570 B2 · Progressive snapshots in automated software testing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,570 B2 - Document ID: US9519570B2 - Status: Granted - Kind: B2 - Application number: 14/219,050 - Filed: 2014-03-19 - Published: 2015-09-24 - Granted: 2016-12-13 - Earliest claimed: 2014-03-19 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Balaji Srinivasan; Madhu Tadiparthi - Web page: https://patentgrep.com/patent/US9519570B2 ## CPC - G06F 11/3698: Environments for analysis, debugging or testing of software - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Embodiments of the present invention disclose a method, computer program product, and system for recording and displaying graphical user interface snapshots during automated testing. A computer captures a first snapshot of a complete graphical user interface, wherein the graphical user interface has at least one graphical user interface control. The computer determines a location of the at least one graphical user interface control. The computer determines that a first test step has altered the at least one graphical user interface control. The computer captures a second snapshot of the altered graphical user interface control. The computer stores the second snapshot and the location of the graphical user interface control in association with the test step. ## Claims ### Claim 1 A method for progressive snapshot generation in automated testing, the method comprising: capturing, by one or more computer processors, a first snapshot of a complete graphical user interface, wherein the graphical user interface has at least one graphical user interface control; determining, by one or more computer processors, a location of the at least one graphical user interface control; determining, by one or more computer processors, that a first test step has altered the at least one graphical user interface control; capturing, by one or more computer processors, a second snapshot of only a graphical user interface control modified by the first test step; associating the captured second snapshot of only the graphical user interface control with a test step N, where N is an integer; storing, by one or more computer processors, the second snapshot and the location of the graphical user interface control in association with the test step, wherein a rendering program generates a current image and an expected image for each recorded test step, wherein the expected image for each recorded test step comprises the first snapshot of the complete graphical user interface and a snapshot for each subsequent test step N, up to N−1, displayed over the first snapshot of the complete graphical user interface at the location of the graphical user interface control effected by each test step; and displaying, via the rendering program, a highlighted GUI control of the test step N, wherein the highlighted GUI control of the test step N comprises only the GUI control modified by the first test step overlaying the graphical user interface control at the location of the graphical user interface control, such that a user sees only the GUI control modified by the first test step. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving, by the rendering program, an instruction to advance a display of the complete graphical user interface by one step; displaying, by one or more computer processors, the first snapshot of the complete graphical user interface; and overlaying, by one or more computer processors, the second snapshot over the first snapshot at the determined location of the at least one graphical user interface control, in order to generate a current image for the subsequent test step, N+1. ### Claim 3 (depends on 2) The method of claim 2, further comprising: indicating on the display of the first snapshot, the determined location of the at least one graphical user interface control. ### Claim 4 (depends on 1) The method of claim 1, wherein determining the information detailing the location and geometry of the at least one graphical user interface comprises: generating, by one or more computer processors, an event listener, wherein the event listener receives an event message when a pre-event action occurs; in response to receiving an event message, detecting, by the event listener, the at least one graphical user interface control which is being interacted with by the pre-event action; and determining, by one or more computer processors, the location of the at least one graphical user interface control based at least in part on application code associated with the graphical user interface. ### Claim 5 (depends on 1) The method of claim 1, further comprising: determining, by one or more computer processors, that a second test step instructs the graphical user interface to reload the complete graphical user interface; capturing, by one or more computer processors, a third snapshot, wherein the third snapshot is of the complete graphical user interface; associating, by one or more computer processors, the third snapshot with the second test step; and overlaying, by one or more computer processors, the third snapshot over the second snapshot at the determined location of the at least one graphical user interface control within the complete graphical user interface, wherein the resulting image is the expected image for a second test step N, and the current image for a third test step, N+1, where N is equal to 2. ### Claim 6 (depends on 1) The method of claim 1, wherein the test step comprises detecting a pre-event action, wherein the pre-event action comprises an event which occurs directly prior to the alteration of the at least one graphical user interface control; and detecting a post-event action, wherein the post-event action comprises a second event which occurs directly after the alteration to the at least one graphical user interface control and concludes the test step, and wherein an event listener detects the pre-event action and the post-event action, and wherein the pre-event action comprises at least one of: down clicking a mouse button, pressing a key on a keyboard, and placing a finger on a touch screen, and the post-event action comprises at least one of: up clicking a mouse button, releasing a key on a keyboard, and removing a finger from a touch screen. ### Claim 7 (depends on 1) The method of claim 1, wherein the second snapshot is stored in association with metadata, and wherein the information detailing the location and geometry of the graphical user interface are stored in the associated metadata. ### Claim 8 A computer program product for optimizing fields in object oriented programming, the computer program product comprising: one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising: program instructions to capture a first snapshot of a complete graphical user interface, wherein the graphical user interface has at least one graphical user interface control; program instructions to determine a location of the at least one graphical user interface control; program instructions to determine that a first test step has altered the at least one graphical user interface control; program instructions to capture a second snapshot of only a graphical user interface control modified by the first test step; program instructions to associate the captured second snapshot of only the graphical user interface control with a test step N, where N is an integer; program instructions to store the second snapshot and the location of the graphical user interface control in association with the test step, wherein a rendering program generates a current image and an expected image for each recorded test step, wherein the expected image for each recorded test step comprises the first snapshot of the complete graphical user interface and a snapshot for each subsequent test step N, up to N−1, displayed over the first snapshot of the complete graphical user interface at the location of the graphical user interface control effected by each test step; and program instructions to display, via the rendering program, a highlighted GUI control of the test step N, wherein the highlighted GUI control of the test step N comprises only the GUI control modified by the first test step overlaying the graphical user interface control at the location of the graphical user interface control, such that a user sees only the GUI control modified by the first test step. ### Claim 9 (depends on 8) The computer program product of claim 8, further comprising: program instructions to receive, by the rendering program, an instruction to advance a display of the complete graphical user interface by one step; program instructions, stored on the one or more computer readable storage media, to display the first snapshot of the complete graphical user interface; and program instructions, stored on the one or more computer readable storage media, to overlay the second snapshot over the first snapshot at the determined location, in order to generate a current image for the subsequent test step, N+1. ### Claim 10 (depends on 9) The computer program product of claim 9, further comprising: program instructions, stored on the one or more computer readable storage media, to indicate on the display of the first snapshot, the determined location. ### Claim 11 (depends on 8) The computer program product of claim 8, wherein the program instructions to determine the information detailing the location and geometry of the at least one graphical user interface comprises: program instructions to generate an event listener, wherein the event listener receives an event message when a pre-event action occurs; program instructions to, in response to receiving an event message, detect, by the event listener, the at least one graphical user interface control which is being interacted with by the pre-event action; and program instructions to determine the location of the at least one graphical user interface control based at least in part on application code associated with the graphical user interface. ### Claim 12 (depends on 8) The computer program product of claim 8, further comprising: program instructions, stored on the one or more computer readable storage media, to determine that a second test step instructs the graphical user interface to reload the complete graphical user interface; program instructions, stored on the one or more computer readable storage media, to capture a third snapshot, wherein the third snapshot is of the complete graphical user interface; program instructions, stored on the one or more computer readable storage media, to associate, by one or more computer processors, the third snapshot with the second test step; and program instructions, stored on the one or more computer readable storage media, to overlay the third snapshot over the second snapshot at the determined location of the at least one graphical user interface control within the complete graphical user interface, wherein the resulting image is the expected image for a second test step N, and the current image for a third test step, N+1, where N is equal to 2. ### Claim 13 (depends on 8) The computer program product of claim 8, wherein the test step comprises detecting a pre-event action, wherein the pre-event action comprises an event which occurs directly prior to the alteration of the at least one graphical user interface control; and detecting a post-event action, wherein the post-event action comprises a second event which occurs directly after the alteration to the at least one graphical user interface control and concludes the test step, and wherein an event listener detects the pre-event action and the post-event action, and wherein the pre-event action comprises at least one of: down clicking a mouse button, pressing a key on a keyboard, and placing a finger on a touch screen, and the post-event action comprises at least one of: up clicking a mouse button, releasing a key on a keyboard, and removing a finger from a touch screen. ### Claim 14 (depends on 8) The computer program product of claim of claim 8, wherein the second snapshot is stored in association with metadata, and wherein the information detailing the location and geometry of the graphical user interface are stored in the associated metadata. ### Claim 15 A computer system for optimizing fields in object oriented programming, the computer system comprising: one or more computer processors; one or more computer readable storage media; program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the program instructions comprising: program instructions to capture a first snapshot of a complete graphical user interface, wherein the graphical user interface has at least one graphical user interface control; program instructions to determine a location of the at least one graphical user interface control; program instructions to determine that a first test step has altered the at least one graphical user interface control; program instructions to capture a second snapshot of only a graphical user interface control modified by the first test step; program instructions to associate the captured second snapshot of only the graphical user interface control with a test step N, where N is an integer; program instructions to store the second snapshot and the location of the graphical user interface control in association with the test step, wherein a rendering program generates a current image and an expected image for each recorded test step, wherein the expected image for each recorded test step comprises the first snapshot of the complete graphical user interface and a snapshot for each subsequent test step N, up to N−1, displayed over the first snapshot of the complete graphical user interface at the location of the graphical user interface control effected by each test step; and program instructions to display, via the rendering program, a highlighted GUI control of the test step N, wherein the highlighted GUI control of the test step N comprises only the GUI control modified by the first test step overlaying the graphical user interface control at the location of the graphical user interface control, such that a user sees only the GUI control modified by the first test step. ### Claim 16 (depends on 15) The computer system of claim 15, further comprising: program instructions to receive, by the rendering program, an instruction to advance a display of the complete graphical user interface by one step; program instructions, stored on the one or more computer readable storage media, to display the first snapshot of the complete graphical user interface; and program instructions, stored on the one or more computer readable storage media, to overlay the second snapshot over the first snapshot at the determined location, in order to generate a current image for the subsequent test step, N+1. ### Claim 17 (depends on 16) The computer system of claim 16, further comprising: program instructions, stored on the one or more computer readable storage media, to indicate on the display of the first snapshot, the determined location. ### Claim 18 (depends on 15) The computer system of claim 15, wherein the program instructions to determine the information detailing the location and geometry of the at least one graphical user interface comprises: program instructions to generate an event listener, wherein the event listener receives an event message when a pre-event action occurs; program instructions to, in response to receiving an event message, detect, by the event listener, the at least one graphical user interface control which is being interacted with by the pre-event action; and program instructions to determine the location of the at least one graphical user interface control based at least in part on application code associated with the graphical user interface. ### Claim 19 (depends on 15) The computer system of claim 15, further comprising: program instructions, stored on the one or more computer readable storage media, to determine that a second test step instructs the graphical user interface to reload the complete graphical user interface; program instructions, stored on the one or more computer readable storage media, to capture a third snapshot, wherein the third snapshot is of the complete graphical user interface; program instructions, stored on the one or more computer readable storage media, to overlay the third snapshot over the second snapshot at the determined location of the at least one graphical user interface control within the complete graphical user interface, wherein the resulting image is the expected image for a second test step N and the current image for a third test step, N+1, where N is equal to 2. ### Claim 20 (depends on 15) The computer system of claim 15, wherein the test step comprises detecting a pre-event action, wherein the pre-event action comprises an event which occurs directly prior to the alteration of the at least one graphical user interface control; and detecting a post-event action, wherein the post-event action comprises a second event which occurs directly after the alteration to the at least one graphical user interface control and concludes the test step, and wherein an event listener detects the pre-event action and the post-event action, and wherein the pre-event action comprises at least one of: down clicking a mouse button, pressing a key on a keyboard, and placing a finger on a touch screen, and the post-event action comprises at least one of: up clicking a mouse button, releasing a key on a keyboard, and removing a finger from a touch screen.