# US 12,111,803 B2 · Combining user feedback with an automated entity-resolution process executed on a computer system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,111,803 B2 - Document ID: US12111803B2 - Status: Granted - Kind: B2 - Application number: 18/299,973 - Filed: 2023-04-13 - Published: 2024-04-18 - Granted: 2024-10-08 - Earliest claimed: 2022-10-14 - Assignees: SAS Institute Inc. - Applicants: SAS Institute Inc. - Inventors: Nicholas Ablitt - Web page: https://patentgrep.com/patent/US12111803B2 ## CPC - G06F 16/215: Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors - G06F 16/2365: {Ensuring data consistency and integrity} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract One example described herein involves a system that can receive a set of data records and execute an automated entity resolution (AER) process configured to assign the set of data records to a set of entities. For each entity in the set of entities, the system can generate a respective consistency score for the entity, generate a respective confidence score for the entity based on the respective consistency score for the entity, and determine a respective visual indicator based on the respective confidence score for the entity. The respective visual indicator can indicate a risk of record misassignment to a user. The system can then generate a graphical user interface that includes the respective visual indicator for each of the entities. ## Claims ### Claim 1 A non-transitory computer-readable medium comprising program code that is executable by one or more processors for causing the one or more processors to perform operations including: generating a graphical user interface that includes a graphical element representing an entity, wherein the graphical user interface also includes a visual indicator associated with the graphical element, the visual indicator being configured to indicate a first risk of one or more data records being misassigned to the entity by an automated entity resolution (AER) system; receiving, through the graphical user interface, a user validation of the one or more data records, wherein the user validation is a validation by a user that the one or more data records were correctly assigned to the entity by the AER system; in response to receiving the user validation, updating the same visual indicator to indicate a second risk of the one or more data records being misassigned to the entity by AER system, the second risk being lower than the first risk based on the user validation; and dynamically modifying the same visual indicator associated with the entity in the graphical user interface over a time period based on changes to a set of data records assigned to the entity by the AER system over the time period, wherein the modified same visual indicator indicating a risk of the changed set of data records being misassigned to the entity by the AER system. ### Claim 2 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the operations further comprise: performing an AER process using the AER system to assign the one or more data records to the entity, wherein the AER process involves executing a machine-learning model to output a probability metric; determining a confidence score based on the probability metric; and selecting the visual indicator, from among a plurality of visual indicator options, based on the confidence score. ### Claim 3 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the operations further comprise: in response to receiving a new data record that is different from the one or more data records, updating the visual indicator to indicate a third risk of the new data record being misassigned to the entity by the automated entity resolution (AER) system. ### Claim 4 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the operations further comprise: receiving a first selection, from the user and via the graphical user interface, of the graphical element; in response to receiving the first selection, displaying a set of data records corresponding to the entity in the graphical user interface; receiving a second selection, from the user and via the graphical user interface, of a data record in the set of data records; and splitting the data record from the set of data records, wherein the splitting involves removing the data record from the set of data records and reassigning the data record to another entity. ### Claim 5 (depends on 4) The non-transitory computer-readable medium of claim 4, wherein the operations further comprise: based on splitting the data record from the set of data records, updating the visual indicator to indicate a third risk of the set of data records being misassigned to the entity. ### Claim 6 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the operations further comprise: receiving a selection, from the user and via the graphical user interface, of a data record that is not assigned to the entity; and assigning the data record to the entity, by including the data record in a set of data records assigned to the entity. ### Claim 7 (depends on 6) The non-transitory computer-readable medium of claim 6, wherein the operations further comprise: based on assigning the data record to the entity, updating the visual indicator to indicate a third risk of the set of data records being misassigned to the entity. ### Claim 8 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the operations further comprise: determining a confidence score associated with the entity; selecting the visual indicator, from among a plurality of visual indicator options, based on the confidence score; determining that the confidence score is below a predefined threshold; and in response to determining that the confidence score is below the predefined threshold, output a warning notification to the user. ### Claim 9 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the operations further comprise: prior to receiving the user validation: receiving a selection of the graphical element by the user; and in response to receiving the selection, displaying a set of data records corresponding to the entity in the graphical user interface, wherein the set of data records includes the one or more data records, and wherein the one or more data records are flagged in the graphical user interface as not yet having been validated by the user; and subsequent to displaying the set of data records, receiving the user validation of the one or more data records. ### Claim 10 (depends on 1) The non-transitory computer-readable medium of claim 1, wherein the operations further comprise: receiving a selection, from the user and via the graphical user interface, of the graphical element; in response to receiving the selection, displaying a set of data records corresponding to the entity in the graphical user interface, wherein at least one data record in the set of data records is flagged for further review by the user. ### Claim 11 A computer-implemented method comprising: generating a graphical user interface that includes a graphical element representing an entity, wherein the graphical user interface also includes a visual indicator associated with the graphical element, the visual indicator being configured to indicate a first risk of one or more data records being misassigned to the entity by an automated entity resolution (AER) system; receiving, through the graphical user interface, a user validation of the one or more data records, wherein the user validation is a validation by a user that the one or more data records were correctly assigned to the entity by the AER system; in response to receiving the user validation, updating the same visual indicator to indicate a second risk of the one or more data records being misassigned to the entity by AER system, the second risk being lower than the first risk based on the user validation; and dynamically modifying the same visual indicator associated with the entity in the graphical user interface over a time period based on changes to a set of data records assigned to the entity by the AER system over the time period, wherein the modified same visual indicator indicating a risk of the changed set of data records being misassigned to the entity by the AER system. ### Claim 12 (depends on 11) The method of claim 11, further comprising: performing an AER process using the AER system to assign the one or more data records to the entity, wherein the AER process involves executing a machine-learning model to output a probability metric; determining a confidence score based on the probability metric; and selecting the visual indicator, from among a plurality of visual indicator options, based on the confidence score. ### Claim 13 (depends on 11) The method of claim 11, further comprising: in response to receiving a new data record that is different from the one or more data records, updating the visual indicator to indicate a third risk of the new data record being misassigned to the entity by the automated entity resolution (AER) system. ### Claim 14 (depends on 11) The method of claim 11, further comprising: receiving a first selection, from the user and via the graphical user interface, of the graphical element; in response to receiving the first selection, displaying a set of data records corresponding to the entity in the graphical user interface; receiving a second selection, from the user and via the graphical user interface, of a data record in the set of data records; and splitting the data record from the set of data records, wherein the splitting involves removing the data record from the set of data records and reassigning the data record to another entity. ### Claim 15 (depends on 14) The method of claim 14, further comprising: based on splitting the data record from the set of data records, updating the visual indicator to indicate a third risk of the set of data records being misassigned to the entity. ### Claim 16 (depends on 11) The method of claim 11, further comprising: receiving a selection, from the user and via the graphical user interface, of a data record that is not assigned to the entity; and assigning the data record to the entity, by including the data record in a set of data records assigned to the entity. ### Claim 17 (depends on 16) The method of claim 16, further comprising: based on assigning the data record to the entity, updating the visual indicator to indicate a third risk of the set of data records being misassigned to the entity. ### Claim 18 (depends on 11) The method of claim 11, further comprising: determining a confidence score associated with the entity; selecting the visual indicator, from among a plurality of visual indicator options, based on the confidence score; determining that the confidence score is below a predefined threshold; and in response to determining that the confidence score is below the predefined threshold, output a warning notification to the user. ### Claim 19 (depends on 11) The method of claim 11, further comprising: prior to receiving the user validation: receiving a selection of the graphical element by the user; and in response to receiving the selection, displaying a set of data records corresponding to the entity in the graphical user interface, wherein the set of data records includes the one or more data records, and wherein the one or more data records are flagged in the graphical user interface as not yet having been validated by the user; and subsequent to displaying the set of data records, receiving the user validation of the one or more data records. ### Claim 20 (depends on 11) The method of claim 11, further comprising: receiving a selection, from the user and via the graphical user interface, of the graphical element; in response to receiving the selection, displaying a set of data records corresponding to the entity in the graphical user interface, wherein at least one data record in the set of data records is flagged for further review by the user. ### Claim 21 A system comprising: one or more processors; and one or more memories comprising program code that is executable by the one or more processors for causing the one or more processors to perform operations including: generating a graphical user interface that includes a graphical element representing an entity, wherein the graphical user interface also includes a visual indicator associated with the graphical element, the visual indicator being configured to indicate a first risk of one or more data records being misassigned to the entity by an automated entity resolution (AER) system; receiving, through the graphical user interface, a user validation of the one or more data records, wherein the user validation is a validation by a user that the one or more data records were correctly assigned to the entity by the AER system; in response to receiving the user validation, updating the same visual indicator to indicate a second risk of the one or more data records being misassigned to the entity by AER system, the second risk being lower than the first risk based on the user validation; and dynamically modifying the same visual indicator associated with the entity in the graphical user interface over a time period based on changes to a set of data records assigned to the entity by the AER system over the time period, wherein the modified same visual indicator indicating a risk of the changed set of data records being misassigned to the entity by the AER system. ### Claim 22 (depends on 21) The system of claim 21, wherein the operations further comprise: performing an AER process using the AER system to assign the one or more data records to the entity, wherein the AER process involves executing a machine-learning model to output a probability metric; determining a confidence score based on the probability metric; and selecting the visual indicator, from among a plurality of visual indicator options, based on the confidence score. ### Claim 23 (depends on 21) The system of claim 21, wherein the operations further comprise: in response to receiving a new data record that is different from the one or more data records, updating the visual indicator to indicate a third risk of the new data record being misassigned to the entity by the automated entity resolution (AER) system. ### Claim 24 (depends on 21) The system of claim 21, wherein the operations further comprise: receiving a first selection, from the user and via the graphical user interface, of the graphical element; in response to receiving the first selection, displaying a set of data records corresponding to the entity in the graphical user interface; receiving a second selection, from the user and via the graphical user interface, of a data record in the set of data records; and splitting the data record from the set of data records, wherein the splitting involves removing the data record from the set of data records and reassigning the data record to another entity. ### Claim 25 (depends on 24) The system of claim 24, wherein the operations further comprise: based on splitting the data record from the set of data records, updating the visual indicator to indicate a third risk of the set of data records being misassigned to the entity. ### Claim 26 (depends on 21) The system of claim 21, wherein the operations further comprise: receiving a selection, from the user and via the graphical user interface, of a data record that is not assigned to the entity; and assigning the data record to the entity, by including the data record in a set of data records assigned to the entity. ### Claim 27 (depends on 26) The system of claim 26, wherein the operations further comprise: based on assigning the data record to the entity, updating the visual indicator to indicate a third risk of the set of data records being misassigned to the entity. ### Claim 28 (depends on 21) The system of claim 21, wherein the operations further comprise: determining a confidence score associated with the entity; selecting the visual indicator, from among a plurality of visual indicator options, based on the confidence score; determining that the confidence score is below a predefined threshold; and in response to determining that the confidence score is below the predefined threshold, output a warning notification to the user. ### Claim 29 (depends on 21) The system of claim 21, wherein the operations further comprise: prior to receiving the user validation: receiving a selection of the graphical element by the user; and in response to receiving the selection, displaying a set of data records corresponding to the entity in the graphical user interface, wherein the set of data records includes the one or more data records, and wherein the one or more data records are flagged in the graphical user interface as not yet having been validated by the user; and subsequent to displaying the set of data records, receiving the user validation of the one or more data records. ### Claim 30 (depends on 21) The system of claim 21, wherein the operations further comprise: receiving a selection, from the user and via the graphical user interface, of the graphical element; in response to receiving the selection, displaying a set of data records corresponding to the entity in the graphical user interface, wherein at least one data record in the set of data records is flagged for further review by the user.