# US 12,117,928 B2 · Unitary test protocols for software program applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,117,928 B2 - Document ID: US12117928B2 - Status: Granted - Kind: B2 - Application number: 17/861,149 - Filed: 2022-07-08 - Published: 2024-01-11 - Granted: 2024-10-15 - Earliest claimed: 2022-07-08 - Assignees: T-Mobile USA, Inc. - Applicants: T-Mobile USA, Inc. - Inventors: Raju Chavan; Aaron Haehn; Aarti Pachauri - Web page: https://patentgrep.com/patent/US12117928B2 ## CPC - G06F 11/3692: {for test results analysis} - G06F 11/3688: {for test execution, e.g. scheduling of test suites} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks - G06N 3/09: Supervised learning - G06N 5/022: Knowledge engineering; Knowledge acquisition - G06N 20/00: Machine learning - G06F 2218/12: Classification; Matching - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A test orchestration engine can create a unitary test configured to perform multiple tests of an application program with a single journey. The engine can identify actions performable by the application program and determine, based on a machine learning (ML) model, those actions that are performable sequentially to validate a set of features of the application program. The system creates the unitary test protocol including a directed sequence of actions to validate the set of features for the test protocols. The system then causes execution of the unitary test protocol to generate a result indicative of performance of the application program. ## Claims ### Claim 1 A computer-readable storage medium, excluding transitory signals and carrying instructions, which, when executed by at least one data processor of a system, cause the system to: instantiate a session for a test orchestration engine to perform a scope of testing that is equivalent of multiple tests for an application program on a user device; identify multiple actions performable by the application program on the user device, wherein the multiple actions are performable in respective journeys of separate and distinct test protocols configured to validate a set of features of the application program, and wherein each journey includes a sequence of one or more actions; determine, based on a machine learning (ML) model, a set of actions of the multiple actions of at least two journeys that are non-mutually exclusive and are performable sequentially to validate the set of features of the application program, wherein the ML model is generated based on a knowledgebase of test protocols and verifications for multiple features of one or more application programs, and wherein the ML model is trained based on multiple journeys on the one or more application programs that perform analogous actions on multiple user devices; create a unitary test protocol including a directed sequence for the set of actions to validate the set of features for the separate and distinct test protocols with a single journey of the unitary test protocol; and cause execution of the unitary test protocol to generate a result indicative of performance of the application program based on performance of the single journey including the directed sequence of the set of actions. ### Claim 2 (depends on 1) The computer-readable storage medium of claim 1, wherein the system is further caused to: output, on a display device of the system, an indication of the result, wherein the indication of the result includes a status for each of the set of features. ### Claim 3 (depends on 1) The computer-readable storage medium of claim 1, wherein the system is further caused to: generate an additional unitary test protocol that defines an additional unitary sequence for performing another group of actions to validate another set of features of the separate and distinct test protocols and an additional test protocol; and cause execution of the additional unitary test protocol to generate an additional result indicative of performance of the application program in response to performance of the additional unitary sequence, wherein the additional result is caused to display on a display device of the system. ### Claim 4 (depends on 1) The computer-readable storage medium of claim 1, wherein the separate and distinct test protocols comprise two or more of: a unit test protocol configured to test one or more software modules of the application program and associated control data, a usage procedure, or an operating procedure to determine fit for use, an integration test protocol configured to test integration of one or more software modules or subsystems associated with the application program to evaluate compliance with a specified functional requirement, a system test protocol configured to test a complete system that supports the one or more software modules or subsystems to evaluate compliance with a specified functional requirement, and an end-to-end test protocol configured to test functionality and performance of the application program under a product-like circumstance. ### Claim 5 (depends on 1) The computer-readable storage medium of claim 1, wherein the system is further caused to: train the ML model based on the result indicative of performance of the application program based on performance of the single journey including the directed sequence of the set of actions; and terminate the session for a test orchestration engine. ### Claim 6 (depends on 1) The computer-readable storage medium of claim 1: wherein the user device is a wireless device of a telecommunications network, and wherein the set of features of the application program includes a payment capability, a capability to execute a purchase operation, and a capability to add a service of the telecommunications network to a subscription of the wireless device. ### Claim 7 (depends on 1) The computer-readable storage medium of claim 1, wherein to identify the multiple actions performable by the application program comprises causing the system to: isolate actions of the journeys of the separate and distinct testing protocols; and evaluate an effect of each action on verification of a feature of the application program. ### Claim 8 (depends on 1) The computer-readable storage medium of claim 1, wherein the multiple actions include user interactions with controls presented by the application program on a display device of user devices, and wherein the multiple actions are configured to control performance of an operation associated with a feature of the application program. ### Claim 9 (depends on 1) The computer-readable storage medium of claim 1, wherein the ML model is trained based on multiple journeys performed on the application program on the user device or multiple user devices. ### Claim 10 (depends on 1) The computer-readable storage medium of claim 1, wherein the ML model is trained based on multiple journeys performed on different application programs on the user device or multiple user devices. ### Claim 11 (depends on 1) The computer-readable storage medium of claim 1, wherein to generate the unitary test protocol comprises causing the system to: determine an order for the set of actions of the unitary sequence as a directed acyclic journey on the application program; and generate the unitary test protocol as an executable journey to perform the directed sequence on the application program. ### Claim 12 A test engine comprising: at least one hardware processor; and at least one non-transitory memory storing instructions, which, when executed by the at least one hardware processor, cause the test engine to: identify multiple actions of multiple journeys for testing an application program, wherein each journey includes a sequence of one or more actions; select a set of actions of at least two journeys the multiple journeys to validate the application program, wherein the set of actions precludes mutually exclusive actions of the multiple journeys; create a unitary test that defines a single journey including the set of actions arranged in a preset order configured to validate a feature of the application program based on performance of the single journey; and cause execution of the unitary test to generate a result indicative of whether the application program passed the unitary test. ### Claim 13 (depends on 12) The test engine of claim 12 further caused to, prior to the multiple actions of the multiple journeys being identified: receive a set of test protocols including a unit test, an integration test, a system test, and an end-to-end test for the application program, wherein the set of test protocols includes the multiple journeys for testing the application program separately for each journey. ### Claim 14 (depends on 12) The test engine of claim 12, wherein to select the set of actions from the multiple journeys to validate the application program comprises causing the test engine to: determine, based on a machine learning (ML) model, the set of actions selected based on historical data including analogous actions of tests for multiple application programs. ### Claim 15 (depends on 12) The test engine of claim 12, wherein to select the set of actions of the multiple journeys to validate the application program comprises causing the test engine to: determine the set of actions selected based on a machine learning (ML) model generated from a supervised knowledgebase. ### Claim 16 (depends on 12) The test engine of claim 12, wherein to select the set of actions of the multiple journeys to validate the application program comprises causing the test engine to: generate an additional unitary test including a combination of the set of actions and an additional action, wherein the additional action is inserted in the ordered set of actions to validate a set of features based on performance of another single journey that includes the additional action. ### Claim 17 (depends on 12) The test engine of claim 12, wherein to select the set of actions of the multiple journeys to validate the application program comprises causing the test engine to: isolate an action of each journey; and evaluate an effect of each isolated action on verification of the feature of the application program. ### Claim 18 A computer-readable storage medium, excluding transitory signals and carrying instructions, which, when executed by at least one data processor of a system, cause the system to: receive a set of test protocols including two or more of a unit test, an integration test, a system test, or an end-to-end test for an application program, wherein each test protocol includes a journey of multiple journeys for separately testing an application program, and wherein each journey includes a sequence of one or more actions; select a set of actions from at least two journeys the multiple journeys to validate an application program, wherein the set of actions precludes mutually exclusive actions of the multiple journeys; generate a unitary test that orders the set of actions to validate a set of features of the application program with a single journey; and cause execution of the unitary test to generate a result indicative of whether the application program passes the set of test protocols based on performance of the single journey. ### Claim 19 (depends on 18) The computer-readable storage medium of claim 18, wherein to generate the unitary test protocol comprises causing the system to: determine an order for the set of actions for the single journey as a directed acyclic journey on the application program. ### Claim 20 (depends on 18) The computer-readable storage medium of claim 18, wherein the system is further caused to: output, on a display device of the system, an indication of the result, wherein the indication of the result includes a status for each of the set of test protocols.