# US 12,001,318 B2 · Self sustained regression testing framework > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,318 B2 - Document ID: US12001318B2 - Status: Granted - Kind: B2 - Application number: 16/237,108 - Filed: 2018-12-31 - Published: 2020-07-02 - Granted: 2024-06-04 - Earliest claimed: 2018-12-31 - Assignees: Paypal, Inc. - Applicants: PayPal, Inc. - Inventors: Prasanth Kuricheti; Jogendar Singh; Rajesh Kumar; Xinran Fang; Naresh Kumar Paturi; Durjay Kumar Mishra; Anil Kumar Kandru; Venkata Siva Sai Manepalli - Web page: https://patentgrep.com/patent/US12001318B2 ## CPC - G06F 11/3698: Environments for analysis, debugging or testing of software - G06F 11/3688: {for test execution, e.g. scheduling of test suites} - G06F 11/3692: {for test results analysis} - G06F 11/3696: {Methods or tools to render software testable} - H04L 9/3236: {using cryptographic hash functions} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Systems, methods, and computer program products for testing new software are provided. Multiple payloads that correspond to scenarios in a production computing environment are identified. From the multiple payloads unique payloads are identified. User data that corresponds to the unique payloads is created. A first testing environment conducts a test using software components in the production environment, the unique payloads, and the user data to generate expected results. A second testing environment conducts a test using new software that replaces at least one of the software components in the production environment, the unique payloads, and the user data, to generate actual results. The one or more attributes in the expected results are compared to the one or more attributes in the actual results to determine if the new software causes an error. ## Claims ### Claim 1 A method comprising: identifying a plurality of payloads that correspond to scenarios in a production computing environment; determining hashes for the plurality of payloads by: identifying a subset of attributes in a plurality of attributes in each payload in the plurality of payloads, wherein the subset of attributes include data from the scenarios in the production computing environment; and determining a hash of the identified subset of attributes in the each payload; aggregating payloads in the plurality of payloads that have a same first hash in the hashes into a first set of payloads; aggregating payloads in the plurality of payloads that have a same second hash in the hashes into a second set of payloads; selecting a first payload from the first set of payloads aggregated according to the first hash and a second payload from the second set of payloads aggregated according to the second hash into a set of unique payloads, wherein the first payload includes first values of the subset of attributes and the second payload includes second values of the subset of attributes different from the first values; creating user data, wherein the user data is associated with the set of unique payloads; testing the set of unique payloads with the user data in a first testing environment to generate expected results, wherein the first testing environment includes software components in the production computing environment; testing the set of unique payloads with the user data in a second testing environment to generate actual results, wherein the second testing environment includes the software components in the production computing environment and new software; comparing the expected results with the actual results; and identifying an error in the new software based on the comparing. ### Claim 2 (depends on 1) The method of claim 1, wherein aggregating the payloads into the first set of payloads further comprises: aggregating the payloads that have the same first hash and a same third hash in the hashes into the first set of payloads. ### Claim 3 (depends on 1) The method of claim 1, wherein determining the hashes further comprises: determining the same first hash in the hashes using the subset of attributes associated with the first payload; and determining the same second hash in the hashes using the subset of attributes associated with the second payload, wherein the subset of attributes are same types of attributes in the plurality of payloads that have different values in the first payload and the second payload. ### Claim 4 (depends on 1) The method of claim 1, further comprising: removing a first type of data from the set of unique payloads, wherein the first type of data includes at least user information. ### Claim 5 (depends on 1) The method of claim 1, further comprising: storing the user data in a memory cache accessible to the first testing environment and the second testing environment. ### Claim 6 (depends on 1) The method of claim 1, wherein the comparing further comprises: configuring at least one attribute in the expected results generated by the first testing environment and the actual results generated by the second testing environment; and comparing a value of the at least one attribute in the expected results to a value of the at least one attribute in the actual results. ### Claim 7 (depends on 6) The method of claim 6, further comprising: determining that the value in the expected results is different from the value in the actual results; and identifying the error in the second testing environment based on the determining. ### Claim 8 (depends on 1) The method of claim 1, wherein the comparing further comprises: configuring a first attribute in the expected results generated by the first testing environment and a second attribute in the actual results generated by the second testing environment; comparing a first value of the first attribute in the expected results to a first value of the second attribute in the actual results; comparing a second value of the first attribute in the expected results to a second value of the second attribute in the actual results; and determining the error when the first value of the first attribute in the expected results matches the first value of the second attribute in the actual results and the second value of the first attribute in the expected results does not match the second value of the second attribute in the actual results. ### Claim 9 (depends on 1) The method of claim 1, wherein creating the user data further comprises: identifying a payload in the plurality of payloads; identifying user production data that is associated with the payload; and creating the user data that emulates the user production data. ### Claim 10 A system comprising: a non-transitory memory storing instructions; and one or more hardware processors coupled to the non-transitory memory and configured to read the instructions from the non-transitory memory to cause the system to perform operations comprising: identifying a plurality of payloads that correspond to scenarios in a production computing environment, wherein each payload includes a request and a response for a transaction that passes through a distribute computing system; determining hashes for the plurality of payloads by: identifying multiple attributes in a plurality of attributes in each payload in the plurality of payloads, wherein the multiple attributes include data from the transaction; and determining a hash of the identified multiple attributes in the each payload; aggregating payloads in the plurality of payloads that have a same first hash in the hashes into a first set of payloads; aggregating payloads in the plurality of payloads that have a same second hash in the hashes into a second set of payloads; selecting a first payload from the first set of payloads aggregated according to the first hash and a second payload from the second set of payloads aggregated according to the second hash into a set of unique payloads, wherein the first payload includes values of the multiple attributes different from values of the multiple attributes in the second payload; creating user data, wherein the user data is associated with the set of unique payloads; testing the set of unique payloads with the user data in a first testing environment to generate expected results, wherein the first testing environment includes software components in the production computing environment; testing the set of unique payloads with the user data in a second testing environment to generate actual results, wherein the second testing environment includes the software components in the production computing environment and new software; comparing the expected results with the actual results; and identifying an error in the new software based on the comparing. ### Claim 11 (depends on 10) The system of claim 10, wherein the operations further comprise: aggregating payloads that have the same first hash and a same third hash in the hashes into the first set of payloads. ### Claim 12 (depends on 11) The system of claim 11, wherein the operations that determine the hashes further comprise: determining the same first hash in the hashes using the multiple attributes associated with the first payload; and determining the same second hash in the hashes using the multiple attributes associated with the second payload, wherein the multiple attributes are same types of attributes in the plurality of payloads and have different values in the first hash and the second hash. ### Claim 13 (depends on 10) The system of claim 10, wherein the operations further comprise: removing a first type of data from the set of unique payloads, wherein the first type of data includes at least user information. ### Claim 14 (depends on 10) The system of claim 10, wherein the operations further comprise: storing the user data in a memory cache accessible to the first testing environment and the second testing environment. ### Claim 15 (depends on 10) The system of claim 10, wherein the operations further comprise: configuring at least one attribute in the expected results generated by the first testing environment and the actual results generated by the second testing environment; and comparing a value of the at least one attribute in the expected results to a value of the at least one attribute in the actual results. ### Claim 16 (depends on 15) The system of claim 15, wherein the operations further comprise: determining that the value in the expected results is different from the value in the actual results; and identifying the error in the second testing environment based on the determining. ### Claim 17 (depends on 10) The system of claim 10, wherein the operations that compare further comprise: configuring a first attribute in the expected results generated by the first testing environment and a second attribute in the actual results generated by the second testing environment; comparing a first value of the first attribute in the expected results to a first value of the second attribute in the actual results; comparing a second value of the first attribute in the expected results to a second value of the second attribute in the actual results; and determining the error when the first value of the first attribute in the expected results matches the first value of the second attribute in the actual results and the second value of the first attribute in the expected result actual results does not match the second value of the second attribute in the actual results. ### Claim 18 (depends on 10) The system of claim 10, wherein the operations that create the user data further comprise: identifying a payload in the plurality of payloads; identifying user production data that is associated with the payload; and creating user data that emulates the user production data. ### Claim 19 A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations comprising: identifying a plurality of payloads that correspond to scenarios in a production computing environment; determining hashes for the plurality of payloads by: identifying a subset of attributes in each payload in the plurality of payloads, wherein the subset of attributes include data from the production computing environment; and determining a hash of the identified subset of attributes in the each payload; aggregating payloads in the plurality of payloads that have a same first hash in the hashes into a first set of payloads; aggregating payloads in the plurality of payloads that have a same second hash in the hashes into a second set of payloads; selecting a first payload from the first set of payloads aggregated according to the first hash and a second payload from the second set of payloads aggregated according to the second hash into a set of unique payloads, wherein the first payload includes values of the subset of attributes different from values of the subset of attributes included in the second payload; creating user data, wherein the user data is associated with the set of unique payloads; testing the set of unique payloads with the user data in a first testing environment to generate expected results, wherein the first testing environment includes software components in the production computing environment; testing the set of unique payloads with the user data in a second testing environment to generate actual results, wherein the second testing environment includes the software components in the production computing environment and new software that modifies at least one software component in the software components in the production environment; comparing the expected results with the actual results; and identifying an error in the new software based on the comparing. ### Claim 20 (depends on 19) The non-transitory machine-readable medium of claim 19, wherein the operations further comprise: aggregating payloads that have the same first hash and a same third hash in the hashes into the first set of payloads.