# US 11,907,205 B1 · Generic parity solution for highly dynamic sources > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,205 B1 - Document ID: US11907205B1 - Status: Granted - Kind: B1 - Application number: 18/157,869 - Filed: 2023-01-23 - Granted: 2024-02-20 - Earliest claimed: 2023-01-23 - Assignees: Intuit Inc. - Applicants: Intuit Inc. - Inventors: Suman Ghosh; Mayur Madnani - Web page: https://patentgrep.com/patent/US11907205B1 ## CPC - G06F 16/2365: {Ensuring data consistency and integrity} - G06F 16/2358: {Change logging, detection, and notification (replication G06F16/27)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Certain aspects of the present disclosure provide techniques for efficient data parity. Embodiments include receiving, by a first data consuming component, from a publication service, a plurality of database change records indicating changes to an underlying data source. Embodiments include updating, by the first data consuming component, a secondary data store based on the plurality of database change records. Embodiments include selecting a sample subset of the plurality of database change records based on one or more conditions. Embodiments include, for each respective database change record in the sample subset that has already been consumed by a second data consuming component that updated a primary data store based on one or more database change records: comparing the respective database change record to first data from the primary data store that corresponds to the respective database change record and making a parity determination based on the comparing. ## Claims ### Claim 1 A method of efficient data parity, comprising: receiving, by a first data consuming component in a first data pipeline a plurality of database change records from a publication service, the plurality of database change records indicating changes to an underlying data source; updating, by the first data consuming component, a secondary data store to include the plurality of database change records; selecting a sample subset of the plurality of database change records based on one or more conditions; and for each respective database change record in the sample subset that has already been consumed by a second data consuming component in a second data pipeline that received one or more database change records from the publication service and updated a primary data store based on the one or more database change records: comparing the respective database change record to first data from the primary data store that corresponds to the respective database change record; determining an inconsistency between the primary data store and a particular database change record based on the comparing; responsive to determining the inconsistency, determining whether the second data consuming component has consumed a database change record with a higher offset value than the first data consuming component; responsive to determining the second data consuming component has consumed a database change record with a higher offset value than the first data consuming component, delaying a parity determination for the particular database change record in the sample subset until the first data consuming component has consumed the database change record with the higher offset value; and responsive to determining the second data consuming component has consumed no database change record with a higher offset value than the first data consuming component, determining a parity failure, wherein a notification is generated based on determination of the parity failure. ### Claim 2 (depends on 1) The method of claim 1, wherein the primary data store is used by one or more data consumers as an alternate data source to the underlying data source. ### Claim 3 (depends on 1) The method of claim 1, wherein the comparing of the respective database change record to the first data from the primary data store that corresponds to the respective database change record comprises comparing only columns that the respective database change record indicates to be changed. ### Claim 4 (depends on 1) The method of claim 1, wherein the notification indicates a database column for which parity failed. ### Claim 5 (depends on 1) The method of claim 1, wherein the selecting of the sample subset of the plurality of database change records is based on attributes of the plurality of database change records. ### Claim 6 (depends on 5) The method of claim 5, wherein the attributes of the plurality of database change records comprise user identifiers associated with the plurality of database change records. ### Claim 7 (depends on 6) The method of claim 6, wherein the selecting comprises choosing all database change records of the plurality of database change records that are associated with a given user identifier. ### Claim 8 A system, comprising: one or more processors; and a memory comprising instructions that, when executed by the one or more processors, cause the system to: receive, by a first data consuming component in a first data pipeline a plurality of database change records from a publication service, the plurality of database change records indicating changes to an underlying data source; update, by the first data consuming component, a secondary data store to include the plurality of database change records; select a sample subset of the plurality of database change records based on one or more conditions; and for each respective database change record in the sample subset that has already been consumed by a second data consuming component in a second data pipeline that received one or more database change records from the publication service and updated a primary data store based on the one or more database change records: compare the respective database change record to first data from the primary data store that corresponds to the respective database change record; determine an inconsistency between the primary data store and a particular database change record based on the comparing; in response to determining the inconsistency, determine whether the second data consuming component has consumed a database change record with a higher offset value than the first data consuming component; in response to determining the second data consuming component has consumed a database change record with a higher offset value than the first data consuming component, delay a parity determination for the particular database change record in the sample subset until the first data consuming component has consumed the database change record with the higher offset value; and in response to determining the second data consuming component has consumed no database change record with a higher offset value than the first data consuming component, determine a parity failure, wherein a notification is generated based on determination of the parity failure. ### Claim 9 (depends on 8) The system of claim 8, wherein the primary data store is used by one or more data consumers as an alternate data source to the underlying data source. ### Claim 10 (depends on 8) The system of claim 8, wherein the comparing of the respective database change record to the first data from the primary data store that corresponds to the respective database change record comprises comparing only columns that the respective database change record indicates to be changed. ### Claim 11 (depends on 8) The system of claim 8, wherein the notification indicates a database column for which parity failed. ### Claim 12 (depends on 8) The system of claim 8, wherein the selecting of the sample subset of the plurality of database change records is based on attributes of the plurality of database change records. ### Claim 13 (depends on 12) The system of claim 12, wherein the attributes of the plurality of database change records comprise user identifiers associated with the plurality of database change records. ### Claim 14 (depends on 13) The system of claim 13, wherein the selecting comprises choosing all database change records of the plurality of database change records that are associated with a given user identifier. ### Claim 15 A non-transitory computer readable medium comprising instructions that, when executed by one or more processors of a computing system, cause the computing system to: receive, by a first data consuming component in a first data pipeline, a plurality of database change records from a publication service, the plurality of database change records indicating changes to an underlying data source; update, by the first data consuming component, a secondary data store based on the plurality of database change records; select a sample subset of the plurality of database change records based on one or more conditions; and for each respective database change record in the sample subset that has already been consumed by a second data consuming component in a second data pipeline that received one or more database change records from the publication service and updated a primary data store based on the one or more database change records: compare the respective database change record to first data from the primary data store that corresponds to the respective database change record; determine an inconsistency between the primary data store and a particular database change record based on the comparing; in response to determining the inconsistency, determine whether the second data consuming component has consumed a database change record with a higher offset value than the first data consuming component; in response to determining the second data consuming component has consumed a database change record with a higher offset value than the first data consuming component, delay a parity determination for the particular database change record in the sample subset until the first data consuming component has consumed the database change record with the higher offset value; and in response to determining the second data consuming component has consumed no database change record with a higher offset value than the first data consuming component, determine a parity failure, wherein a notification is generated based on determination of the parity failure. ### Claim 16 (depends on 15) The non-transitory computer readable medium of claim 15, wherein the primary data store is used by one or more data consumers as an alternate data source to the underlying data source. ### Claim 17 (depends on 15) The non-transitory computer readable medium of claim 15, wherein the comparing of the respective database change record to the first data from the primary data store that corresponds to the respective database change record comprises comparing only columns that the respective database change record indicates to be changed.