Forming consistency groups using self-describing record sets for remote data duplexing

A remote data shadowing system including a primary site and a secondary site, the secondary site asynchronously shadowing record updates of the primary site in real time for disaster recovery purposes, the record updates generated by applications running at the primary site, the primary site comprising:

• a sysplex timer;
• a primary processor running the applications generating the record updates and issuing a corresponding write I/O operation for each record update, the primary processor having a primary data mover therein;
• a plurality of primary storage controllers directed to store the record updates, the plurality of primary storage controllers executing the issued write I/O operation for each record update;
• and
• a plurality of primary storage devices receiving and storing the record updates therein according to the corresponding write I/O operations,
• wherein the primary processor and each write I/O are time-stamped by the primary processor, as synchronized by the sysplex timer, such that write I/O operations are accurately sequence ordered relative to each other, the primary data mover collecting sets of record updates and combining each record set information as provided by each one of the plurality of primary storage controllers with the corresponding record update, each record set information including a relative sequence and time of each corresponding write I/O operation, the primary data mover collecting record updates into time interval groups and inserting a prefix header to each time interval group, wherein the prefix header includes information identifying the record updates included in each time interval group, each record set information and prefix header combined for creating self describing record sets, the self describing record sets being transmitted to the secondary site, wherein the self describing record sets provide information adequate for the secondary site to shadow the record updates therein in sequence consistent order without further communications from the primary site.

An asynchronous remote data duplexing system providing storage based, real time data shadowing, including a primary site running applications generating record updates and having a secondary site, remote from the primary site, the secondary site shadowing the record updates and providing disaster recovery for the primary site, the asynchronous remote data duplexing system comprising:

• a sysplex timer for synchronizing time dependent processes in the primary site;
• a primary processor at the primary site for running the applications and issuing write I/O operations for corresponding record updates, the primary processor having a primary data mover therein;
• a plurality of primary storage controllers receiving the a write I/O operation for each record update, each primary storage controller write I/O operation synchronized to the sysplex timer by the primary processor;
• a plurality of primary storage devices storing the record updates therein according to the corresponding write I/O operation,
• wherein the primary data mover collects record set information from the plurality of primary storage controllers for each record update and appends a prefix header to a predetermined grouped of record set informations, the prefix header and predetermined record set information groups forming self describing record sets, each record set information including a primary device address, a cylinder number and head number (CCHH), a record update sequence number, a write I/O type, a search argument, a sector number, and a record update time, and wherein the prefix header includes a total data length, an operational time stamp, a time interval group number, and a records read time;
• a secondary processor at the secondary site having a secondary data mover, the secondary data mover receiving the self describing record sets from the primary site;
• a plurality of secondary storage controllers coupled to the secondary processor;
• and
• a plurality of secondary storage devices storing the record updates,
• wherein the secondary data mover determines whether the transmitted self describing record sets are complete and forms consistency groups from the self describing record sets and provides the record updates from each consistency group to the plurality of secondary storage controllers for writing to the plurality of secondary storage devices in an order consistent with a sequence that the record updates were written to the plurality of primary storage devices.