# US 2021/0334239 A1 · System and Method for Re-Synchronizing a Portion of or an Entire Source Database and a Target Database > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0334239 A1 - Document ID: US20210334239A1 - Status: Published application - Kind: A1 - Application number: 16/860,007 - Filed: 2020-04-27 - Published: 2021-10-28 - Earliest claimed: 2020-04-27 - Inventors: RICHARD BANISTER - Web page: https://patentgrep.com/patent/US20210334239A1 ## CPC - G06F 16/178: Techniques for file synchronisation in file systems - G06F 11/0757: {by exceeding a time limit, i.e. time-out, e.g. watchdogs} - G06F 11/1451: {by selection of backup contents} - G06F 11/1464: {for networked environments} - G06F 11/1471: involving logging of persistent data for recovery - G06F 16/113: {Details of archiving (lifecycle management in storage systems G06F3/0649; point-in-time backing up or restoration of persistent data G06F11/1446)} - G06F 16/2379: {Updates performed during online database operations; commit processing} - G06F 16/258: {Data format conversion from or to a database} - G06F 2201/835: Timestamp - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system and method to ensure the consistency of a data warehouse or backup database with a source database are described. The method alleviates issues of comparing two sets of the same data on disparate network systems and eliminates having to reload the entire target database or compare every field to ensure reasonable consistency of the contents. The process involves loading a unique record identifier, an optional record change timestamp, and an optional record archive field of a source database into a work file or temporary database table. Source work file records or temporary database tables records that do not exist in the target database or have timestamp mismatches are retrieved from the source database and added to or updated in the target database. Target database records that are archived or missing in the work file or temporary database table are archived or deleted from the target database. ## Claims ### Claim 1 A computer-implemented method comprising: a. Storing a source database in a computer system, wherein a plurality of source records of the source database each include a date time stamp of a most recent revision and a unique record identifier, wherein each record identifier is a unique signifier within the source database and a record creation event is a type of record revision; b. Storing a target database in a transient, persistent memory, wherein the target database receives copies of revisions of the plurality of source records; c. Selecting within the target database of a plurality of records to update; d. Generating a nonpersistent source work file comprising source compound keys extracted from each current source record of the plurality of source records, and each source compound key extracted from a same source record and including a source record identifier and a most recent source date time stamp of the same source record; e. Sorting the source compound keys of the nonpersistent source work file into a nonpersistent source ordered list according to a value sequence of individual source record identifiers of the records from which the compound keys are generated; f. Generating a nonpersistent target work file comprising target compound keys extracted from each current target record of the plurality of target records, and each target compound key is extracted from an individual target record and including a target record identifier and a most recent target date time stamp of a same target record; g. Sorting the target compound keys of the nonpersistent target work file into a nonpersistent target ordered list according to the value sequence of the record identifiers of the target compound keys as applied to order the nonpersistent source ordered list of the nonpersistent source work file; h. Comparing the nonpersistent source work file and the nonpersistent target work file to generate a nonpersistent retrieval work file that lists source compound keys not found to be matched in the nonpersistent target work file by a target compound key having a same target record identifier and target date time stamp, wherein records listed as source compound keys in the nonpersistent source work file and not present as target compound keys in the nonpersistent target work file are added to the nonpersistent retrieval work file; and i. Writing all source records having corresponding source compound keys listed in the nonpersistent retrieval work file into the target database; wherein record updating received by the target database from the source database is unidirectional. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein records listed as source compound keys in the source work file and not present as target compound keys in the target work file are added to the retrieval work file. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein records referenced by target compound keys in the target work file and not present as source compound keys in the source work file have an archive field set to a deletion indicator value. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprising deleting any target record having a unique compound key presented in a source record when the source record comprises an archive field having a deletion indicator value. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein each source record has an archive field and no source compound key of a source record having any archive field set to a deletion indicator value is written into the nonpersistent source work file. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein each source record has an archive field and only source compound keys of source record having any archive field set to an active record indicator value are written into the nonpersistent source work file. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the source record identifier may be a compound key comprising information sourced from two or more data fields of the comprising source record comprising the source record identifier. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the target record identifier may be a compound key comprising information sourced from two or more data fields of the target record comprising the target record identifier. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the plurality of source records is assigned to individual threads of the multithreaded process in view of each date time stamp of the individual source record. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the source database is a database selected from the group of database types comprising a relational database, a NOSQL database, a graph database, an object-oriented database, an XML code, a JSON code, one or more flat digitized software files, and one or more Software-as-a Service databases accessible via a communications network. ### Claim 11 (depends on 1) The computer-implemented method of claim 1, wherein each source compound key extracted from the source database has a date time stamp corresponding to a time no earlier than indicated by a pre-selected date time value. ### Claim 12 (depends on 1) The computer-implemented method of claim 1, wherein at least one target record has an archive field indicating that a corresponding source record is deleted. ### Claim 13 (depends on 16) The computer-implemented method of claim 16, wherein the at least one target record further comprises additional source data record field content of the corresponding source record that was present at the time of the corresponding source record being deleted from the source database. ### Claim 14 (depends on 1) The computer-implemented method of claim 1, wherein the computer memory storing the target database is integral to the computer system. ### Claim 15 A database management system, comprising: a processor comprising hardware; and a transient persistent memory storing a database management system software adapted to update a first database with information received via an electronic communications network and the memory further storing executable instructions that, when executed by the processor, perform operations comprising: a. Storing a source database in a computer system, wherein a plurality of source records of the source database each include a date time stamp of a most recent revision and a unique record identifier, wherein each record identifier is a unique signifier within the source database and a record creation event is a type of record revision; b. Storing a target database in a computer memory, wherein the target database receives copies of revisions of the plurality of source records; c. Selecting within the target database of a plurality of records to update; d. Generating a nonpersistent source work file comprising source compound keys extracted from each current source record of the plurality of source records, and each source compound key extracted from a same source record and including a source information and a most recent source date time stamp of the same source record; d. Sorting the source compound keys of the nonpersistent source work file into a nonpersistent source ordered list according to a value sequence of individual source record identifiers of the source records; e. Generating a nonpersistent target work file comprising target compound keys extracted from each current target record of the plurality of target records, and each target compound key extracted from a same target record and including a target information and a most recent target date time stamp of the same target record; f. Sorting the target compound keys of the nonpersistent target work file into a nonpersistent target ordered list according to the value sequence of individual record identifiers of the target records as applied to order the source ordered list of the source work file; g. Comparing the nonpersistent source work file and the nonpersistent target work file to generate a nonpersistent retrieval work file that lists source compound keys not found to be matched in the nonpersistent target work file by a target compound key having a same target record identifier and target date time stamp; and h. Writing all source records having corresponding source compound keys listed in the nonpersistent retrieval work file into the target database. ### Claim 16 (depends on 15) The database management system of claim 15, wherein records listed as source compound keys in the source work file and not present as target compound keys in the target work file are deleted from the target database. ### Claim 17 (depends on 15) The database management system of claim 15, wherein records listed as target compound keys in the nonpersistent target work file and not present as source compound keys in the nonpersistent source work file have an archive field set to a deletion indicator value. ### Claim 18 (depends on 15) The database management system of claim 15, wherein the computer memory storing the target database is integral to the database management system. ### Claim 19 (depends on 15) The database management system of claim 15, wherein the transient persistent memory storing the target database is external to the database management system. ### Claim 20 A database management system, the database management system bi-directionally communicatively coupled with a computer system that stores a source database, wherein a plurality of source records of the source database each include a date time stamp of a most recent revision and a unique record identifier, and each record identifier is a unique signifier within the source database, the database management system comprising: a processor comprising hardware; and a memory storing a database management system software adapted to update a first database with information received via an electronic communications network and the memory further storing executable instructions that, when executed by the processor, perform operations comprising: a. Storing a target database in a computer memory, wherein the target database receives copies of revisions of the plurality of source record and stores the revisions of the plurality of source records as target records, and a record creation event is processed as a type of record revision; b. Generating a new or truncated nonpersistent temporary table from a prior comparison operation; c. Writing at least one key and at least one archive value for each record in the source database into the nonpersistent temporary table; d. Comparing all of the keys of active records of the nonpersistent temporary table with a SQL statement to determine which records referenced by the nonpersistent temporary table are missing or marked as archived in the target database table; e. Generating a nonpersistent refresh list of the record identifiers of each target database record determined to require an update; f. Downloading all records associated with a record identifier of the nonpersistent refresh list from the source database and adding each downloaded record to the target database; and g. Comparing records in the nonpersistent temporary table and the target database table to determine which records that are marked archived in the nonpersistent temporary table are showing as active in the target database table by checking all active records in the target database table against the list of active records in the nonpersistent temporary table. ### Claim 21 (depends on 20) The database management system of claim 20, wherein records not listed in the nonpersistent temporary table and present in the target table are deleted from the target database. ### Claim 22 (depends on 20) The database management system of claim 20, wherein records not listed in the nonpersistent temporary table and present in the target table have an archive field set to a deletion indicator value. ### Claim 23 (depends on 22) The database management system of claim 22, wherein the executable instructions further direct the processor to perform additional operations of writing each timestamp from the source database for each record associated with a record identifier in the nonpersistent temporary table into the nonpersistent temporary table prior to comparing all of the keys in the target database table with a SQL statement to determine which records are missing from the target database or marked as archived in the target database. ### Claim 24 (depends on 20) The database management system of claim 20, wherein the executable instructions still further direct the processor to perform additional operations of comparing all of the keys in the target database table with a SQL statement to determine which records are missing from the target database or marked as archived in the target database. ### Claim 25 (depends on 20) The database management system of claim 20, wherein the executable instructions further direct the processor to perform additional operations of comparing time stamps associated with a target database table record and nonpersistent temporary table entry associated with the same key to determine whether the record in the target database is current, or whether the source database contains a newer version of the record. ### Claim 26 (depends on 20) The database management system of claim 20, wherein the executable instructions further direct the processor to perform additional operations comprising: a. Applying a fixed or configurable or automatically determined starting time to filter the records in the source database table into the nonpersistent temporary table; b. Generating an ancillary list of record identifiers of added or changed records from only a plurality of active records captured in the nonpersistent temporary table to add or update the record data in the target table; and c. Downloading the source records associated of record identifiers of the ancillary list, and inserting or updating the corresponding target database records associated of record identifiers in the target table with the newly downloaded records. ### Claim 27 (depends on 26) The method of claim 26, wherein the executable instructions further direct the processor to perform additional operations of marking each target record of the target database referenced in the target table as archived when a same record identifier of the nonpersistent temporary table is marked as archived. ### Claim 28 (depends on 27) The method of claim 27, wherein the executable instructions further direct the processor to perform the method upon a designated subset of the records in the target database. ### Claim 29 A computer-implemented method comprising: a. Storing a source database in a computer system, wherein a plurality of source records of the source database each include a source update indicator of a most recent update and a unique record identifier, wherein each record identifier is a unique signifier within the source database and a record creation event is a type of record revision; b Storing a target database in a transient, persistent memory, wherein the target database receives copies of revisions of the plurality of source records; c. Selecting within the target database of a plurality of records to update; d. Generating a nonpersistent source work file comprising source compound keys extracted from each current source record of the plurality of source records, and each source compound key extracted or derived from a same source record and including the source record identifier and the source update indicator of the most recent update of the same source record; e. Sorting the source compound keys of the nonpersistent source work file into a nonpersistent source ordered list according to a value sequence of individual source record identifiers of the records from which the compound keys are generated; f. Generating a nonpersistent target work file comprising target compound keys extracted from each current target record of the plurality of target records, and each target compound key is extracted from an individual target record and including a target record identifier and a target update indicator of a most recent update of a same target record; g. Sorting the target compound keys of the nonpersistent target work file into a nonpersistent target ordered list according to the value sequence of the record identifiers of the target compound keys as applied to order the nonpersistent source ordered list of the nonpersistent source work file; h. Comparing the nonpersistent source work file and the nonpersistent target work file to generate a nonpersistent retrieval work file that lists source compound keys not found to be matched in the target work file by a target compound key having a same target record identifier and target update indicator; and i. Writing all source records having corresponding source compound keys listed in the retrieval work file into the target database. ### Claim 30 (depends on 29) The method of claim 29, wherein the computer system performs the method upon a designated subset of the records in the target database. ### Claim 31 (depends on 1) The method of claim 1, wherein a memory storage for the computer system storing the nonpersistent source ordered list, the nonpersistent target ordered list, or the nonpersistent retrieval work file ordered list is selected from the memory group consisting of disk storage, block storage, system memory, or a database table. ### Claim 32 (depends on 1) The method of claim 1, wherein the transient, persistent memory storing the target database is selected from the memory group consisting of disk storage, block storage, system memory, or a database table.