# US 9,436,579 B2 · Real-time, multi-tier load test results aggregation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,579 B2 - Document ID: US9436579B2 - Status: Granted - Kind: B2 - Application number: 12/804,338 - Filed: 2010-07-19 - Published: 2012-01-19 - Granted: 2016-09-06 - Earliest claimed: 2010-07-19 - Assignees: SOASTA, Inc. - Applicants: Tal Broda; Michael Hemmert; Matthew Solnit - Inventors: Tal Broda; Michael Hemmert; Matthew Solnit - Web page: https://patentgrep.com/patent/US9436579B2 ## CPC - G06F 11/3495: {for systems} - G06F 11/3409: {for performance assessment} - G06F 11/3414: {Workload generation, e.g. scripts, playback} - G06F 11/3419: {by assessing time} - G06F 2201/875: Monitoring of systems including the internet - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for real-time analysis of results from a load test performed on a target website includes periodically computing first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website. The first-level aggregated test results are computed from data received by each of the load server instances from the target website every first time interval. The first-level aggregated test results are then periodically sent from each of the load server instances to an associated one of a plurality of analytic server instances every second time interval. The first-level aggregated test results are aggregated by each of the analytic server instances to produce second-level aggregated test results, which test results may then further aggregated to produce third-level aggregated test results at a data storage instance in real-time. ## Claims ### Claim 1 A computer-implemented method for real-time analysis of results from a load test as the load test is being performed on a target website, comprising: (a) computing first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website, the load including requests of a plurality of virtual users, the first-level aggregated test results including first-level response time statistics computed from data received by each of the plurality of load server instances from the target website in response to the generated load, the first-level aggregated test results including statistics computed by an embedded component within each of the plurality of load server instances every first time interval, the statistics including an average response time across a subset of the plurality of virtual users; (b) sending the first-level aggregated test results from each of the plurality of load server instances to an associated one of a plurality of analytic server instances every second time interval, the second time interval being longer than the first time interval; (c) computing second-level aggregated test results by each of the plurality of analytic server instances in response to receiving the first-level aggregated test results from the plurality of load server instances every second time interval, the second-level aggregated test results including second-level response time statistics computed from the first-level aggregated test results; (d) storing the second-level aggregated test results in a data storage instance; (e) running a query to the data storage instance to perform one or more statistical functions on the second-level aggregated test results, thereby producing third-level aggregated test results, the third-level aggregated test results including a website response time statistic across all of the virtual users; and (f) communicating the third-level aggregated test results to a browser application for providing an analytic dashboard with a graphical user interface that outputs the website response time statistic, wherein steps (a)-(f) are each performed in real-time as the load test is being performed on the target website. ### Claim 2 (depends on 1) The computer-implemented method of claim 1 wherein the plurality of load server instances and the plurality of analytic server instances are provided by at least one cloud provider. ### Claim 3 (depends on 1) The computer-implemented method of claim 1 wherein the first time interval is approximately one second. ### Claim 4 (depends on 1) The computer-implemented method of claim 1 wherein the second time interval is approximately five seconds. ### Claim 5 (depends on 1) The computer-implemented method of claim 1 further comprising changing, in real-time responsive to user input, a time format of a chart displayed on the analytic dashboard of the third-level aggregated test results. ### Claim 6 (depends on 1) The computer-implemented method of claim 1 wherein the periodically sending of the first-level aggregated test results comprises periodically sending a batch message to the associated one of the plurality of analytic server instances, the batch message comprising all of the first-level aggregated test results computed during the second time interval. ### Claim 7 A non-transitory computer-readable storage medium encoded with a computer program product, which, when executed, is operable to: (a) compute first-level aggregated test results within each of a plurality of load server instances during execution of a load test that generates a load on a target website, the load including requests of a plurality of virtual users, the first-level aggregated test results including first-level response time statistics computed from data received by each of the plurality of load server instances from the target website in response to the generated load, the first-level aggregated test results including statistics computed by an embedded component within each of the plurality of load server instances every first time interval, the statistics including an average response time across a subset of the plurality of virtual users; (b) send the first-level aggregated test results from each of the plurality of load server instances to an associated one of a plurality of analytic server instances every second time interval, the second time interval being longer than the first time interval; (c) compute second-level aggregated test results by each of the plurality of analytic server Instances in response to receiving the first-level aggregated test results from the plurality of load server instances every second time interval, the second-level aggregated test results including second-level response time statistics computed from the first-level aggregated test results; (d) store the second-level aggregated test results in a data storage instance; (e) run a query to the data storage instance to perform one or more statistical functions on the second-level aggregated test results, thereby producing third-level aggregated test results, the third-level aggregated test results including a website response time statistic across all of the virtual users; and (f) communicate the third-level aggregated test results to a browser application for providing an analytic dashboard with a graphical user interface that outputs the website response time statistic, wherein (a)-(f) are each performed in real-time during the running of the load test. ### Claim 8 (depends on 7) The non-transitory computer-readable storage medium of claim 7 wherein the computer program product, when executed, is further operable to periodically send one or more messages to the associated one of the plurality of analytic server instances, each message comprising the first-level aggregated test results computed in one of the first time intervals. ### Claim 9 (depends on 7) The non-transitory computer-readable storage medium of claim 7 wherein the computer program product, when executed, is further operable to store the website response time statistic in the data storage instance, the website response time statistic being available on a real-time basis to the browser application. ### Claim 10 (depends on 7) The non-transitory computer-readable storage medium of claim 7 wherein the plurality of load server instances and the plurality of analytic server instances are provided by more than one cloud provider. ### Claim 11 (depends on 7) The non-transitory computer-readable storage medium of claim 7 wherein the first time interval is approximately one second and the second time interval is approximately five seconds. ### Claim 12 (depends on 7) The non-transitory computer-readable storage medium of claim 7 wherein the computer program product, when executed, is further operable to change, in real-time responsive to user input, a time format of a chart displayed on the analytic dashboard of the third-level aggregated test results. ### Claim 13 A computer-implemented method for real-time analysis of results from a load test as the load test is being performed on a target website, comprising: computing first-level aggregated test results within each of a plurality of load server instances, each load server instance simulating a plurality of virtual users that generate a load on the target website, the first-level aggregated test results including statistics computed by an embedded component within each of the plurality of load server instances as the load test is being performed from data received by each of the plurality of load server instances from the target website in response to the generated load, the first-level aggregated test results being computed every N seconds, where N is a positive integer, the statistics including an average response time across a subset of the plurality of virtual users; sending, as the load test is being performed, a batch message that includes all of the first-level aggregated test results computed by each of the plurality of load server instances to an associated one of a plurality of analytic server instances every M seconds, where M is an integer greater than N; computing, as the load test is being performed, second-level aggregated test results by each of the plurality of analytic server instances in response to receiving the first-level aggregated test results from the plurality of load server instances every M seconds, the second-level aggregated test results including second-level response time statistics computed from the first-level aggregated test results; storing, as the load test is being performed, the second-level aggregated test results in a data storage instance; running, as the load test is being performed, a query to the data storage instance to perform one or more statistical functions on the second-level aggregated test results, thereby producing third-level aggregated test results, the third-level aggregated test results including a website response time statistic across all of the virtual users; and communicating, as the load test is being performed, the third-level aggregated test results to a browser application for providing an analytic dashboard with a graphical user interface that outputs the website response time statistic in real-time. ### Claim 14 (depends on 13) The computer-implemented method of claim 13 wherein the plurality of load server instances and the plurality of analytic server instances are provided by more than one cloud provider.