Data processing system with mechanism for restoring file systems based on transaction logs

A data processing system with a logging mechanism which stores log records for repairing an inconsistent file system, the system comprising:

• a primary storage subsystem;
• a secondary storage subsystem;
• a metadata volume, created in said secondary storage subsystem, which store a plurality of metadata objects describing files;
• a log volume, created in said secondary storage subsystem, which stores log records describing updates made to the metadata objects;
• a metadata cache, created in said primary storage subsystem, which temporarily stores the metadata objects;
• metadata loading means, responsive to a transaction attempting to update metadata objects, for loading the requested metadata objects from said metadata volume to said metadata cache;
• log collection means for collecting log records which indicate what updates were made to the metadata objects in said metadata cache;
• a log buffer which stores the log record collected by said log collection means;
• log writing means for transporting the log records from said log buffer to said log volume at regular intervals, using storage space of said log volume as a circular buffer;
• metadata writing means for writing back the updated metadata objects from said metadata cache to said metadata volume;
• valid log range monitoring means for observing the writing back operation performed by said metadata writing means so as to check which updated metadata object has been written back to said metadata volume, and based on the observation, identifying a particular set of log records stored in said log volume as valid log records, the particular set of log records corresponding to such updated metadata objects that have not yet been written back to said metadata volume;
• and
• file system restoration means, responsive to a file system restoration request, for correcting inconsistencies in the metadata objects stored in said metadata volume by using the valid log records identified by said valid log range monitoring means.

A data processing system with a logging mechanism which stores log records for repairing an inconsistent file system, the system comprising:

• a primary storage subsystem;
• a secondary storage subsystem;
• a metadata volume, created in said secondary storage subsystem, which store a plurality of metadata objects describing files;
• a metadata cache, created in said primary storage subsystem, which temporarily stores the metadata objects;
• metadata loading means, responsive to a transaction attempting to update metadata objects, for loading the requested metadata objects from said metadata volume to said metadata cache;
• and
• log collection means for examining what type of transaction is in progress, to determine whether the current transaction is likely to make a plurality of updates to a single metadata object, and if the current transaction is likely to do so, collecting only the final state of the updated metadata object.

A computer-readable medium storing a computer program for file system management which collects transmission log records for repairing an inconsistent file system, the computer program being designed to run on a computer having primary and secondary storage subsystems in order to cause the computer to function as:

• a plurality of metadata volumes, created in the secondary storage subsystem, which store a plurality of metadata objects describing files;
• a log volume, created in the secondary storage subsystem, which stores log records describing updates made to the metadata objects;
• a metadata cache, created in the primary storage subsystem, which temporarily stores the metadata objects;
• metadata loading means, responsive to each transaction attempting to updated metadata objects, for loading the requested metadata objects from said metadata volume to said metadata cache;
• metadata management means for holding metadata volume location identifiers associated with the metadata objects loading to said metadata cache, the metadata volume location identifiers indicating in which of said metadata volumes the metadata objects were stored;
• log collection means for collecting log records which indicate what updates were made to the metadata objects in said metadata cache, the log records containing the metadata volume location identifiers corresponding to the updated metadata objects;
• a log buffer which stores the log record collected by said log collection means;
• and
• log writing means for transporting the log records from said log buffer to said log volume.

A computer-readable medium storing a computer program for file system management which collects transmission log records for repairing an inconsistent file system, the computer program being designed to run on a computer having primary and secondary storage subsystems in order to cause the computer to function as:

• a metadata volume, created in the secondary storage subsystem, which stores a plurality of metadata objects describing files;
• a log volume, created in the secondary storage subsystem, which stores log records describing updates made to the metadata objects;
• a metadata cache, created in the primary storage subsystem, which temporarily stores the metadata objects;
• metadata loading means, responsive to each transaction attempting to update metadata objects, for loading the requested metadata objects from said metadata volume to said metadata cache;
• log collection means for collecting log records which indicate what updates were made to the metadata objects in said metadata cache;
• a log buffer which stores the log record collected by said log collection means;
• log writing means for transporting the log records from said log buffer to said log volume at regular intervals, using storage space of said log volume as a circular buffer;
• metadata writing means for writing back updated metadata objects from said metadata cache to said metadata volume;
• valid log range monitoring means for observing the writing back operation performed by said metadata writing means so as to check which updated metadata object has been written back to said metadata volume, and based on the observation, identifying a particular set of log records stored in said log volume as valid log records, the particular set of log records corresponding to such updated metadata objects that have not yet been written back to said metadata volume;
• and
• file system restoration means, responsive to a file system restoration request, for correcting inconsistencies in the metadata objects stored in said metadata volume by using the valid log records identified by said valid log range monitoring means.

A computer-readable medium storing a computer program for file system management which collects transmission log records for repairing an inconsistent file system, the computer program being designed to run on a computer having primary and secondary storage subsystems in order to cause the computer to function as:

• a metadata volume, created in the secondary storage subsystem, which store a plurality of metadata objects describing files;
• a metadata cache, created in the primary storage subsystem, which temporarily stores the metadata objects;
• metadata loading means, responsive to each transaction attempting to update metadata objects, for loading the requested metadata objects from said metadata volume to said metadata cache;
• and
• log collection means for examining what type of transaction is in progress, to determine whether the current transaction is likely to make a plurality of updates to a single metadata object, and if the current transaction is likely to do so, collecting only the final state of the updated metadata object.

A computer-readable medium storing a computer program for file system management which collects transmission log records for repairing an inconsistent file system, the computer program being designed to run on a computer having primary and secondary storage subsystems in order to cause the computer to function as:

• allocation management data memory means for holding allocation management data for use in controlling allocation of metadata objects;
• metadata allocation means, responsive to a first transaction requesting allocation of a free metadata object, for searching the allocation management data to find and allocate a free metadata object to the first transaction, and updating the allocation management data so as to indicate that the allocated metadata object is in use;
• metadata deallocation means, responsive to a second transaction requesting deallocation of a specific metadata object being allocated, for updating the allocation management data so as to indicate that the specified metadata object is free;
• and
• log collection means for collecting log records describing which part of the allocation management data was modified by the metadata allocation means and metadata deallocation means;
• wherein each collected log record describes only a single resource allocated or deallocated to/from a single transaction.

A data processing system with a logging mechanism which stores log records for repairing an inconsistent file system, the system comprising:

• allocation management data memory means for holding allocation management data for use in controlling allocation of metadata objects;
• metadata allocation means, responsive to a first transaction requesting allocation of a free metadata object, for searching the allocation management data to find and allocate a free metadata object to the first transaction, and updating the allocation management data so as to indicate that the allocated metadata object is in use;
• metadata deallocation means, responsive to a second transaction requesting deallocation of a specific metadata object being allocated, for updating the allocation management data so as to indicate that the specified metadata object is free;
• and
• log collection means for collecting log records describing which part of the allocation management data was modified by the metadata allocation means and metadata deallocation means;
• wherein each collected log record describes only a single resource allocated or deallocated to/from a single transaction.