# US 2019/0171552 A1 · Test Plan Generation Using Machine Learning > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0171552 A1 - Document ID: US20190171552A1 - Status: Published application - Kind: A1 - Application number: 15/829,126 - Filed: 2017-12-01 - Published: 2019-06-06 - Earliest claimed: 2017-12-01 - Assignees: Sap SE - Applicants: Sap SE - Inventors: John Mitchell; Alok Bhatgunaki; Mohinish Shaikh; Aishwarya Soni - Web page: https://patentgrep.com/patent/US20190171552A1 ## CPC - G06F 11/3688: {for test execution, e.g. scheduling of test suites} - G06F 11/3672: {Test management} - G06F 11/3684: {for test design, e.g. generating new test cases} - G06N 3/09: Supervised learning - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Testing of a software system is initiated in an operating environment. The software system includes a plurality of software programs executing across multiple computing nodes. Thereafter, the operating environment and/or resources available to one or more of the software programs are selectively altered according to a test plan. In addition, functional and/or performance characteristics of one or more parts of the operating environment and/or the software programs under test are also selectively altered according to the test plan. In addition, concurrent with the altering of the operating environment and/or the resources and the altering of functional and/or performance characteristics, behavior and/or performance of the software system are monitored to identify faults. Data characterizing the faults can be provided as input into the testing for analysis and feedback into generating better test plans as well as generating more optimal configurations of the software system itself and/or the operating environment. ## Claims ### Claim 1 A computer-implemented method comprising: initiating, in an operating environment, testing of a software system comprising a plurality of software programs executing across multiple computing nodes; selectively altering resources available to one or more of the software programs according to a test plan; selectively altering functional and/or performance characteristics of one or more parts of the operating environment and/or the software programs under test according to the test plan; monitoring, concurrent with the altering of resources and the altering of functional and/or performance characteristics, behavior and/or performance of the software system to identify faults; and providing data characterizing the faults. ### Claim 2 (depends on 1) The method of claim 1, wherein the providing of data comprising at least one of: causing the data to be displayed in an electronic visual display, transmitting the data to a remote computing device, loading the data into memory, or storing the data in electronic physical persistence. ### Claim 3 (depends on 1) The method of claim 1, wherein the selectively altered resources comprise at least one of: memory, I/O bandwidth, network, processor resources, or hardware capabilities. ### Claim 4 (depends on 1) The method of claim 1, wherein the test plan specifies a sequence of test events which cause the resources for specific software programs to be altered. ### Claim 5 (depends on 4) The method of claim 4, wherein each test event quantifies an amount of variance for the corresponding altered resource. ### Claim 6 (depends on 1) The method of claim 1, wherein the test plan specifies a decision tree of test events in which each test event is triggered based on an occurrence of a pre-defined condition. ### Claim 7 (depends on 1) The method of claim 1 further comprising: receiving data characterizing the software system; and generating, using at least one machine learning model trained using data characterizing testing of a plurality of historical software systems, the test plan that optimally characterizes behavior and/or operational faults associated with the software system. ### Claim 8 (depends on 7) The method of claim 7, wherein the at least one machine learning model comprises at least one of: a neural network model, a logistic regression model, a support vector machine, a random forest, a nearest neighbor model, a Bayesian model, or a genetic algorithm. ### Claim 9 A system comprising: at least one data processor; and memory storing instructions which, when executed by the at least one data processor, result in operations comprising: initiating, in an operating environment, testing of a software system comprising a plurality of software programs executing across multiple computing nodes; selectively altering resources available to one or more of the software programs according to a test plan; selectively altering functional and/or performance characteristics of one or more parts of the operating environment and/or the software programs under test according to the test plan; monitoring, concurrent with the altering of resources and the altering of functional and/or performance characteristics, behavior and/or performance of the software system to identify faults; and providing data characterizing the faults. ### Claim 10 (depends on 9) The system of claim 9, wherein the providing of data comprising at least one of: causing the data to be displayed in an electronic visual display, transmitting the data to a remote computing device, loading the data into memory, or storing the data in electronic physical persistence. ### Claim 11 (depends on 9) The system of claim 9, wherein the selectively altered resources comprise at least one of: memory, I/O bandwidth, network, processor resources, or hardware capabilities. ### Claim 12 (depends on 9) The system of claim 9, wherein the test plan specifies a sequence of test events which cause the resources for specific software programs to be altered. ### Claim 13 (depends on 12) The system of claim 12, wherein each test event quantifies an amount of variance for the corresponding altered resource. ### Claim 14 (depends on 9) The system of claim 9, wherein the test plan specifies a decision tree of test events in which each test event is triggered based on an occurrence of a pre-defined condition. ### Claim 15 (depends on 9) The system of claim 9, wherein the operations further comprise: receiving data characterizing the software system; and generating, using at least one machine learning model trained using data characterizing testing of a plurality of historical software systems, the test plan that optimally characterizes behavior and/or operational faults associated with the software system. ### Claim 16 (depends on 15) The system of claim 15, wherein the at least one machine learning model comprises at least one of: a neural network model, a logistic regression model, a support vector machine, a random forest, a nearest neighbor model, a Bayesian model, or a genetic algorithm. ### Claim 17 A non-transitory computer program product storing instructions which, when executed by at least one data processor forming part of at least one computing device, results in operations comprising: initiating, in an operating environment, testing of a software system comprising a plurality of software programs executing across multiple computing nodes; selectively altering resources available to one or more of the software programs according to a test plan; selectively altering functional and/or performance characteristics of one or more parts of the operating environment and/or the software programs under test according to the test plan; monitoring, concurrent with the altering of resources and the altering of functional and/or performance characteristics, behavior and/or performance of the software system to identify faults; and providing data characterizing the faults. ### Claim 18 (depends on 17) The computer program product of claim 17, wherein the selectively altered resources comprise at least one of: memory, I/O bandwidth, network, processor resources, or hardware capabilities. ### Claim 19 (depends on 17) The computer program product of claim 17, wherein: the test plan specifies a sequence of test events which cause the resources for specific software programs to be altered; and each test event quantifies an amount of variance for the corresponding altered resource; the test plan specifies a decision tree of the test events in which each test event is triggered based on an occurrence of a pre-defined condition. ### Claim 20 (depends on 17) The computer program product of claim 17, wherein the operations further comprise: receiving data characterizing the software system; and generating, using at least one machine learning model trained using data characterizing testing of a plurality of historical software systems, the test plan that optimally characterizes behavior and/or operational faults associated with the software system; wherein the at least one machine learning model comprises at least one of: a neural network model, a logistic regression model, a support vector machine, a random forest, a nearest neighbor model, a Bayesian model, or a genetic algorithm.