# US 2026/0252477 A1 · Software testing of a policy management system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0252477 A1
- Document ID: US20260252477A1
- Status: Published application
- Kind: A1
- Application number: 19/062,780
- Filed: 2025-02-25
- Published: 2026-08-27
- Earliest claimed: 2025-02-25
- Assignees: State Farm Mutual Automobile Insurance Company
- Applicants: State Farm Mutual Automobile Insurance Company
- Inventors: Jacob Carlson
- Web page: https://patentgrep.com/patent/US20260252477A1

## CPC

- G06F 11/3688: {for test execution, e.g. scheduling of test suites}
- G06F 11/3698: Environments for analysis, debugging or testing of software
- G06F 11/3604: Analysis of software for verifying properties of programs (testing of software G06F11/3668)
- G06F 11/368: {for test version control, e.g. updating test cases to a new software version}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Techniques for implementing regression testing of a software application that interacts with external systems via structured data input/output files are described herein. The systems and techniques described herein allows for verification of differences between a verified and a development version of the software application in a continuous integration (CI) development environment. Additionally, the techniques described herein identify or shortlist code changes that may have caused the differences, and provide a notification to respective developers of the identified code changes. The notifications may indicate the code changes ordered by a likelihood of having caused the differences, and may additionally, provide automatically-generated textual descriptions to aid with the verification.

## Claims

### Claim 1

A software testing system, comprising:
a processor; and
memory storing computer-executable instructions that, when executed by the processor, cause the system to perform operations comprising:
receiving first input data including entries in a graphical user interface (GUI);
providing, to a first software application, at least a portion of the first input data;
receiving, from the first software application and based on the first input data, first output data characterized by a structured hierarchy indicating data fields;
providing, to a second software application, the at least the portion of the first input data;
receiving, from the second software application and based on the at least the portion of the first input data, second output data characterized by the structured hierarchy;
removing, from the first output data and the second output data, data values corresponding to at least a portion of the data fields of the structured hierarchy, wherein removing the data values results in filtered first output data and filtered second output data;
determining a difference between the filtered first output data and the filtered second output data;
determining, based on the difference, one or more changes in source code between the first software application and the second software application; and
providing, to an electronic device, a notification indicating the difference and the one or more changes.

### Claim 2 (depends on 1)

The software testing system of claim 1, wherein the first software application corresponds to a development version of a policy management system, and the second software application corresponds to a production version of the policy management system.

### Claim 3 (depends on 2)

The software testing system of claim 2, wherein the second software application determines the second output data based at least in part on data received from an external system.

### Claim 4 (depends on 1)

The software testing system of claim 1, wherein determining the difference further comprises:
determining that a first data field in the first output data is of a first data type; and
determining that the first data field in the second output data is of a second data type, different from the first data type,
wherein the difference is determined based on the second data type being different from the first data type.

### Claim 5 (depends on 1)

The software testing system of claim 1, the operations further comprising:
determining that a first data field in the first output data comprises a first array with multiple elements arranged in a first order;
determining that the first data field in the second output data comprises a second array with the multiple elements arranged in a second order, different from the first order; and
based on determining that the second output data comprises the multiple elements and the second order is different from the first order, further removing the multiple elements from the first array and the second array in the filtered first output data and the filtered second output data.

### Claim 6 (depends on 1)

The software testing system of claim 1, the operations further comprising:
determining respective likelihoods corresponding to the one or more changes, the likelihoods indicating a certainty level that a respective change caused the difference; and
determining a subset of the one or more changes that correspond to likelihoods greater than a threshold,
wherein the notification presents the subset of the one or more changes and excludes a remainder of the one or more changes.

### Claim 7 (depends on 6)

The software testing system of claim 6, wherein the notification presents the subset of the one or more changes in decreasing order of respective likelihoods.

### Claim 8 (depends on 6)

The software testing system of claim 6, wherein the likelihoods are determined based on inputting at least the respective change and the difference to a trained machine-learned model.

### Claim 9 (depends on 1)

The software testing system of claim 1, wherein:
the notification further includes a textual description of the one or more changes or the difference, and
the textual description is automatically generated by a large language model (LLM) trained on a training set of code changes and resulting differences.

### Claim 10 (depends on 1)

The software testing system of claim 1, further comprising:
receiving, from the electronic device, a verification that the difference is as intended; and
based on receiving the verification, determining that the first software application passes the software testing.

### Claim 11 (depends on 1)

The software testing system of claim 1, further comprising:
receiving, from the electronic device, an indication that the difference is unintended or incorrect; and
based on receiving the indication, determining that the first software application fails the software testing.

### Claim 12

A computer-implemented method, comprising:
receiving a first input data indicating inputs to a software application;
providing, to a development version of the software application, the first input data;
receiving, from the development version of the software application and based on the first input data, a first output data characterized by a structure comprising named data fields and corresponding data values;
providing, to a production version of the software application, the first input data;
receiving, from the production version of the software application and based on the first input data, a second output data characterized by the structure;
removing, from the first output data and the second output data, data values corresponding to at least a portion of the named data fields of the structure, wherein removing the data values results in filtered first output data and filtered second output data;
determining a difference in the named data fields or the corresponding data values between the filtered first output data and the filtered second output data;
determining, based on the difference, one or more changes in source code between the development version of the software application and the production version of the software application; and
providing, to an electronic device, a notification indicating the difference and the one or more changes.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, wherein determining the one or more changes is based at least in part on a timestamp associated with a check-in of respective change to a source code repository.

### Claim 14 (depends on 12)

The computer-implemented method of claim 12, further comprising:
determining respective likelihoods corresponding to the one or more changes, the likelihoods indicating a certainty level that a respective change caused the difference;
determining a subset of the one or more changes that correspond to likelihoods greater than a threshold,
wherein the notification presents the subset of the one or more changes and excludes a remainder of the one or more changes.

### Claim 15 (depends on 12)

The computer-implemented method of claim 12, further comprising:
determining that a first named data field in the first output data comprises a first group of data with multiple elements arranged in a first order;
determining that the first named data field in the second output data comprises a second group of data the multiple elements arranged in a second order, different from the first order; and
based on determining that the second output data comprises the multiple elements and the second order is different from the first order, further removing the multiple elements from the first group of data and the second group of data in the filtered first output data and the filtered second output data.

### Claim 16 (depends on 12)

The computer-implemented method of claim 12, further comprising:
receiving, from the electronic device, a verification that the difference is as intended;
based on receiving the verification, determining that the production version of the software application can be replaced by the development version of the software application.

### Claim 17 (depends on 12)

The computer-implemented method of claim 12, further comprising:
generating, based on an output of a large language model (LLM), a textual description of the one or more changes or the difference,
wherein the notification further includes the textual description.

### Claim 18

A non-transitory computer-readable medium storing instructions which, when executed by a processor causes the processor to:
receive a first input data indicating test inputs to a software application;
provide, to a first version of the software application, the first input data;
receive, from the first version of the software application and based on the first input data, a first output data characterized by a structure comprising data fields and corresponding data values;
provide, to a second version of the software application, the first input data;
receive, from the second version of the software application and based on the first input data, a second output data characterized by the structure;
remove, from the first output data and the second output data, data values corresponding to at least a portion of the data fields of the structure, wherein removing the data values results in filtered first output data and filtered second output data;
determine a difference in the data fields or the corresponding data value between the filtered first output data and the filtered second output data;
determine, based on the difference, one or more changes in source code between the first version of the software application and the second version of the software application; and
provide, to an electronic device, a notification indicating the difference and the one or more changes.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the first version and the second version determine the first output data and the second output data based at least in part on data received from an external system.

### Claim 20 (depends on 18)

The non-transitory computer-readable medium of claim 18, the instructions further causing the processor to:
receive, from the electronic device, a verification that the difference is as intended; and
based on receiving the verification, determine that the first version of the software application passes a regression test with respect to the second version of the software application.
