# US 10,481,969 B2 · Configurable system wide tests > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,481,969 B2 - Document ID: US10481969B2 - Status: Granted - Kind: B2 - Application number: 15/640,272 - Filed: 2017-06-30 - Published: 2018-08-09 - Granted: 2019-11-19 - Earliest claimed: 2017-02-07 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Conal McGarvey; Jakob F. Lichtenberg; Andrew D. Mikesell; Remy L. De Weduwe - Web page: https://patentgrep.com/patent/US10481969B2 ## CPC - G06F 11/079: {Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36)} - G06F 11/0721: {within a central processing unit [CPU]} - G06F 11/263: Generation of test inputs, e.g. test vectors, patterns or sequences {; with adaptation of the tested hardware for testability with external testers} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems and methods to conduct data-driven, system wide tests are disclosed. Instructions for testing an operating system are associated with one or more modules associated with the computing system. When a failed test is identified, a second set of instructions that accounts for the failed test is generated and executed, in order to efficiently identify and target specific issues. Results from these fully configurable testing methods can be communicated and utilized in further applications. ## Claims ### Claim 1 A system comprising: at least one processor; and at least one memory having stored thereon computer executable instructions that, when executed by the at least one processor, cause the system to: receive a first set of instructions comprising information indicative of a first test of one or more modules; cause execution the first test of the one or more modules; receive information identifying a test condition associated with execution of the first test, the test condition associated with a test failure that interrupts execution of the first test or requires intervention before continuing execution of the first test; in response to the information, generate a second set of instructions based on the identified test condition, the second set of instructions indicative of a second test that is determined based at least in part on the identified test condition and modified to bypass the test condition that is associated with the test failure; and cause execution of the second test without resolution of the identified test condition. ### Claim 2 (depends on 1) The system of claim 1, wherein the second set of instructions is based on a permutation of the first set of instructions. ### Claim 3 (depends on 2) The system of claim 2, wherein the permutation excludes the first test. ### Claim 4 (depends on 2) The system of claim 2, wherein the permutation is associated with a module associated with the test condition. ### Claim 5 (depends on 1) The system of claim 1, wherein the second set of instructions includes a plurality of tests associated with the identified test condition. ### Claim 6 (depends on 1) The system of claim 1, wherein the first and second sets of instructions comprise one or more configuration files. ### Claim 7 (depends on 1) The system of claim 1, wherein the first and second sets of instructions comprise an XML file. ### Claim 8 (depends on 1) The system of claim 1, wherein the second set of instructions comprises one or more tests associated with a module associated with the test condition. ### Claim 9 (depends on 1) The system of claim 1, wherein the second set of instructions excludes tests associated with a module associated with the test condition. ### Claim 10 (depends on 1) The system of claim 1, wherein the second set of instructions includes tests associated with a module associated with the test condition. ### Claim 11 A method of performing integration testing of a computing system, the method comprising: receiving, by a computing device, a first set of instructions defining a first plurality of tests, the first plurality of tests associated with testing a plurality of modules of a computer operating system using a test configuration; executing the first plurality of tests based on the first set of instructions; identifying, by the computing device, a test condition associated with executing a first test of the first plurality of tests, the test condition associated with a module of the plurality of modules and associated with a test failure that prevents continued execution of the first test; generating, by the computing device, a second set of instructions based at least in part on the first set of instructions, the second set of instructions defining a second plurality of tests associated with the plurality of modules and modified to bypass the test condition that is associated with the test failure; and executing the second plurality of tests based on the second set of instructions without altering the test configuration. ### Claim 12 (depends on 11) The method of claim 11, wherein the second set of instructions excludes the first test. ### Claim 13 (depends on 11) The method of claim 11, wherein the second set of instructions includes the first test. ### Claim 14 (depends on 11) The method of claim 11, wherein the second set of instructions excludes a second test associated with the module associated with the test condition, the second test selected based at least in part on the test condition. ### Claim 15 (depends on 11) The method of claim 11, wherein the second set of instructions includes a second test associated with the module associated with the test condition, the second test selected based at least in part on the test condition. ### Claim 16 (depends on 11) The method of claim 11, wherein the second set of instructions is generated based at least in part on one or more parameters indicative of testing breadth and testing depth. ### Claim 17 (depends on 11) The method of claim 11, further comprising identifying a minimized set of tests of the plurality of tests for reproducing the test condition. ### Claim 18 A computer-readable storage medium comprising computer-readable instructions that, when executed by a computing device comprising a processor and memory, cause the computing device to perform operations comprising: receiving a first set of instructions defining a first plurality of tests, the first plurality of tests associated with testing a plurality of modules of a computer system in accordance with a test configuration; executing the first plurality of tests based on the first set of instructions; identifying a test condition associated with executing a first test of the first plurality of tests, the test condition associated with the plurality of modules and associated with a failure of the test condition; generating a second set of instructions based at least in part on the first set of instructions, the second set of instructions defining a second plurality of tests associated with the plurality of modules and modified to bypass the failed test condition; and executing the second plurality of tests based on the second set of instructions without resolving the test condition and without altering the test configuration. ### Claim 19 (depends on 18) The computer-readable storage medium of claim 18, wherein the second set of instructions excludes the first test. ### Claim 20 (depends on 18) The computer-readable storage medium of claim 18, further comprising computer-readable instructions that, when executed by a computing device comprising a processor and memory, cause the computing device to perform operations comprising: identifying a minimized set of tests for reproducing the test condition.