Back to results
US 10,474,565 B2Granted
Root cause analysis of non-deterministic tests
AssigneeMicro Focus LLC
InventorsLital Kornfeld, Effi Bar She'an, Talia Rogov
Life of this patent
Estimating the term…
Abstract
Example implementations relate to root cause analysis of non-deterministic tests. In response to detecting a non-deterministic test of an application under test, some implementations may cause rerun executions of the non-deterministic test. Some implementations may also capture data during the rerun executions. The data may include application data, test data, and environment data. Some implementations may also analyze the captured data to determine a root cause of a failure of the non-deterministic test. The analysis may be based on a comparison between failed rerun executions and passed rerun executions.
Claims13
A system for root cause analysis of non-deterministic tests comprising:
- a test rerun execution engine to cause, in response to detecting a non-deterministic test of an application under test, rerun executions of the non-deterministic test;
- a data capture engine to capture data during the rerun executions, the data including application data, test data, and environment data;
- and
- a root cause analysis engine to analyze the captured data to determine a root cause of a failure of the non-deterministic test by identifying patterns in the captured data, the analysis being based on a comparison between failed rerun executions of the rerun executions and passed rerun executions of the rerun executions,
- wherein the patterns are identified based on an analysis of the captured data, the analysis including at least one of:
- a comparison between a number of the failed rerun executions that perform a first particular flow and a number of the passed rerun executions that perform the first particular flow;
- a comparison between a number of the failed rerun executions that perform a second particular flow and pass a first parameter and a number of the passed rerun executions that perform the second particular flow and pass a second parameter;
- a comparison between a size of a first thread count of the failed rerun executions to a size of a second thread count of the passed rerun executions;
- or
- a comparison between an environment related to the failed rerun executions and an environment related to the passed rerun executions.
7 more elements
A computer-implemented method comprising:
- receiving, via a test rerun execution engine, a first request from a first portion of code to capture data during rerun executions of a non-deterministic test of an application under test;
- in response to receiving the first request, and via a data capture engine:
- instrumenting code of the application under test, the instrumented code including instructions to capture the data, the data including metadata related to the application under test and an environment of the application under test;
- and
- executing the instrumented code to capture the data;
- receiving, via a root cause analysis engine, a second request from a second portion of code to analyze the captured data;
- and
- in response to receiving the second request, determining, via the root cause analysis engine, a root cause of a failure of the non-deterministic test by analyzing the captured data to identify patterns in the captured data that are common among failed rerun executions of the non-deterministic test and not common among passed rerun executions of the non-deterministic test,
- wherein the patterns are identified based on an analysis of the captured data, the analysis including at least one of:
- a comparison between a number of the failed rerun executions that perform a first particular flow and a number of the passed rerun executions that perform the first particular flow;
- a comparison between a number of the failed rerun executions that perform a second particular flow and pass a first parameter and a number of the passed rerun executions that perform the second particular flow and pass a second parameter;
- a comparison between a size of a first thread count of the failed rerun executions to a size of a second thread count of the passed rerun executions;
- or
- a comparison between an environment related to the failed rerun executions and an environment related to the passed rerun executions.
11 more elements
A non-transitory machine-readable storage medium including instructions which, when executed by a processor, cause the processor to:
- store captured data in a data repository, the captured data including:
- rerun execution data related to a first set of rerun executions of a non-deterministic test of an application under test, the rerun execution data including execution flows, test status data, and environment data;
- and
- analysis data related to an analysis of the rerun execution data, the analysis data including patterns in the rerun execution data that are common among failed rerun executions and not common among passed rerun executions,
- wherein the patterns are based on an analysis of the rerun execution data, the analysis including at least one of:
- a comparison between a number of the failed rerun executions that perform a first particular flow and a number of the passed rerun executions that perform the first particular flow;
- a comparison between a number of the failed rerun executions that perform a second particular flow and pass a first parameter and a number of the passed rerun executions that perform the second particular flow and pass a second parameter;
- a comparison between a size of a first thread count of the failed rerun executions to a size of a second thread count of the passed rerun executions;
- or
- a comparison between an environment related to the failed rerun executions and an environment related to the passed rerun executions;
- and
- update the analysis data based on a second set of rerun executions of the non-deterministic test.
9 more elements
Dependencies are inferred from the claim text.