System and methods for automated testing of functionally complex systems

A system for automated testing of functionally complex systems, comprising:

• a test manager module operating on a server computer and adapted for managing testing of a distributed functionally complex system comprising functional electronic components of at least two functional types connected across a network and a plurality of networking connectivity components;
• a test data storage subsystem coupled to the test manager module and adapted to store at least test results;
• a log collection module operating on a server computer and adapted to receive log data from a plurality of log collectors;
• an event collection module operating on a server computer and adapted to receive event data from a plurality of event collectors;
• a test execution module operating on a server computer;
• and
• a correlation engine software module operating on a server computer and adapted to perform at least a mathematical correlation analysis of a plurality of test or operating data obtained from one or more of the test execution module, the log collection module, and the event collection module;
• wherein the test manager module causes tests to be executed by the test execution engine, the tests directed to actively testing the functionally complex system for faults;
• further wherein, on detection of an anomalous test result, the test manager causes the correlation engine to analyze data from the log collection module and the event collection module to attempt to positively identify a root cause of the anomalous test result;
• further wherein, when the correlation engine is unable to positively identify a root cause of the anomalous test result, the correlation engine determines at least a plurality of additional tests that will assist the correlation engine in improving correlation results, and the correlation engine causes the test manager to cause the plurality of additional tests to be performed and the results thereof to be sent to the correlation engine in order to enable the correlation engine to positively identify a root cause of the anomalous test result;
• and
• wherein the test manager module causes tests to be executed by the test execution engine prior to the functionally complex system being placed into production in order to detect complex failure modes prior to placing the functionally complex system into production.