Log archive filtering method for transaction-consistent forward recovery from catastrophic media failures

In a transaction-based data processing system having a stable storage resource for storing transaction log records and a first recovery procedure means for database recovery from system failure, said first recovery procedure means including means for executing in a single pass a sequence of recovery log records stored in said stable storage resource as selectable record types including REDO and UNDO types, said system including an archival storage resource for receiving and storing dumps of the contents of said stable storage resource, a method for transaction-consistent resource recovery from failure of said stable storage resource, said method comprising the steps of:

• during forward processing of said transactions, performing in sequence the steps of
• writing transaction log records to a recovery log,
• writing to said recovery log all UNDO type recovery log records created by said first recovery procedure means as log records having a PRE-PSEUDO-CRASH-INTERVAL-UNDO type during a time interval beginning at a predetermined time during said forward processing,
• writing to said recovery log a START-PSEUDO-CRASH type record having a pseudo-crash timestamp value corresponding to the time at the end of said time interval,
• writing to said recovery log a series of PSEUDO-RECOVERY type records in accordance with said first recovery procedure means for recovering from a system failure at the end of said time interval,
• writing to said recovery log and END-PSEUDO-RECOVERY type record having said pseudo-crash timestamp value, and
• writing to said stable storage resource a recovery log copy of all existing recovery log records not already stored in said stable storage resource at completion of said END-PSEUDO-RECOVERY record writing step;
• during an archival dump of said recovery log copy from said stable storage resource to said archival storage resource,
• filtering said recovery log copy records by discarding from one to all UNDO type records while writing all remaining record types to said archival storage resource as filtered recovery log archive records, and
• writing to said stable storage resource a MOST-RECENT-PSEUDO-CRASH record having said pseudo-crash timestamp value in response to writing an END-PSEUDO-RECOVERY record to said archival storage;
• executing said filtered recovery log archive records in accordance with said first recovery procedure means during a single forward pass while ignoring all said PSEUDO-RECOVERY type log archive records associated with any said pseudo-crash timestamp different from the pseudo-crash timestamp contained in said MOST-RECENT-PSEUDO-CRASH record;
• executing said PSEUDO-RECOVERY type log archive records associated with said MOST-RECENT-PSEUDO-CRASH timestamp in accordance with said first recovery procedure means while ignoring all said REDO and UNDO type log archive records made subsequently to the START-PSEUDO-CRASH type log archive record having a timestamp equal to said MOST-RECENT-PSEUDO-CRASH timestamp value;
• and
• executing said PRE-PSEUDO-CRASH-INTERVAL-UNDO type log archive records in the pseudo-crash interval associated with said MOST-RECENT-PSEUDO-CRASH timestamp value in accordance with said first recovery procedure means during a single reverse pass over said most recent recovery log archive pseudo-crash interval.

In a transaction-based data processing system having a stable storage resource for storing transaction log records in blocks denominated "log extents" and a first recovery procedure means for database recovery from system failure, said first recovery procedure means including means for executing in a single pass a sequence of recovery log records stored in said stable storage resource as selectable record types including REDO and UNDO types, said system including an archival storage resource for receiving and storing dumps of the contents of said stable storage resource, a method for transaction-consistent resource recovery from failure of said stable resource, said method comprising the steps of:

• during forward processing of said transactions, performing in sequence the steps of
• writing transaction log records to a recovery log,
• writing to said stable storage resource a recovery log copy of all said recovery log records within a first said log extent not already stored in said stable storage resource at completion of the writing of said first log extent to said recovery log, and
• writing to said stable storage resource an EXTENT-ID type record having a list of all transactions still active at the completion of the writing of said first log extent to said recovery log;
• during an archival dump of said recovery log copy from said stable storage resource to said archival storage resource,
• filtering said recovery log copy records by discarding from one to all UNDO type records except for such UNDO type records involving transactions in said active transaction list while writing all remaining record types to said archival storage resource as filtered recovery log archive records;
• executing said filtered recovery log archive records in accordance with said first recovery procedure means in a single forward pass while ignoring all log archive records following the most recent EXTENT-ID type log archive record;
• and executing said UNDO type log archive records during a single reverse pass over the most recent said log extent in said recovery log archive.

A method, implemented on a digital computer system, for generating in an archival storage resource a filtered recovery log archive having recovery log records of selectable types including UNDO and REDO types generated and written to a recovery log by a first recovery procedure means for resource recovery from system failure, said filtered recovery log archive being for the transaction-consistent resource recovery from a stable storage resource failure in a transaction-based data processing system, said method comprising the steps of:

• writing to said recovery log a START-PSEUDO-CRASH type record having a pseudo-crash timestamp value corresponding to a predetermined pseudo-crash time during the operation of said data processing system;
• writing to said recovery log series of PSEUDO-RECOVERY type records in accordance with said first recovery procedure means for recovering from a system failure occurring as said predetermined pseudo-crash time;
• writing to said recovery log an END-PSEUDO-RECOVERY type record having said pseudo-crash timestamp value;
• writing to said stable storage resource a recovery log copy of all existing recovery log records not already stored in said stable storage resource at completion of said END-PSEUDO-RECOVERY writing step;
• and
• writing said recovery log copy from said stable storage resource to said archival storage resource except for one or more UNDO type recovery log copy records.

A method, implemented on a digital computer system, for generating in an archival storage resource a filtered recovery log archive having recovery records organized in blocks denominated "log extents", said recovery records being of selectable types including UNDO and REDO types generated by a first procedure means for resource recovery from system failure, said filtered recovery log archive being for the transaction-consistent resource recovery from a stable storage resource failure in a transaction-based data processing system, said method comprising the steps of:

• writing to a recovery log a series of recovery records including REDO and UNDO type records in accordance with said first recovery procedure means until all said recovery records in a first said log extent are written to said recovery log;
• writing to said recovery log a list of transactions still active when said first log extent is completely written to said recovery log;
• and
• filtering said recovery log records by discarding from one to all UNDO records not associated with transactions in said active transaction list while writing said recovery log records to said archival storage resource.

A computer system for concurrently executing failure-independent processes having a recovery log containing a plurality of recovery records of different types, including UNDO and REDO types, said system comprising:

• first storage resource means for storing a data resource;
• stable storage resource means for storing data including said recovery log having UNDO and REDO types of said recovery records;
• first recovery procedure means for generating said recovery records in accordance with a first recovery procedure to recover a transaction-consistent version of said data resource after a system failure;
• archival storage resource means for storing a data image dump archive of said data resource and a recovery log archive of said recovery log records;
• pseudo-crash control means for creating a pseudo-crash signal that causes said first recovery means to create a series of pseudo-crash recovery records and for writing said pseudo-crash recovery records to said recovery log;
• archival log filter means for excluding from said recovery log archive from one to all said UNDO type recovery log records except for those generated by said pseudo-crash control means;
• and
• recovery execution means for modifying said data image dump archive in accordance with said first recovery procedure using said recovery log archive records whereby said data image dump archive is brought into a transaction-consistent state.

A computer system for concurrently executing failure-independent processes having a recovery log containing a plurality of recovery records of different types, including UNDO and REDO types, said system comprising:

• first storage resource means for storing a data resource;
• stable storage resource means for storing data including said recovery log having UNDO and REDO types of said recovery records and being organized in blocks denominated "log extents";
• first recovery procedure means for generating said recovery records in accordance with a first recovery procedure to recover a transaction-consistent version of said data resource after a system failure;
• archival storage resource means for storing a data image dump archive of said data resource and a recovery log archive of said recovery log records;
• extent closure means for writing a list of transactions still active when all said recovery records in a first said log extent are written to said recovery log;
• archival log filter means for excluding all UNDO type recovery log records from said recovery log archive except for those related to transactions on said active transaction list;
• and
• recovery execution means for applying said recovery log archive records to said data image dump archive in accordance with said first recovery procedure whereby said data image dump archive is brought into a transaction-consistent state.