Efficient content indexing of incremental block-based backups

A system to detect file changes between block-based backups for indexing contents of an incremental block-based backup of a volume having a file system, the system comprising:

• a processor-based system executed on a computer system and configured to execute instructions comprising:
• allowing the incremental block-based backup of the volume to complete, the incremental backup being a current backup and comprising storing on backup storage all blocks of the volume that have changed since a previous backup according to a changed block list generated by a change block tracking (CBT) driver that intercepts writes from the file system;
• after the incremental backup has completed,
• obtaining the changed block list, the changed block list being a first list;
• reading superblocks of the file system to generate a second list identifying metadata blocks at which metadata records associated with files of the file system are stored;
• intersecting the first and second lists to return a third list identifying changed metadata blocks, the changed metadata blocks thereby corresponding to metadata records that have changed between the previous and current backups;
• reading a changed metadata record, the changed metadata record being a current version of the changed metadata record;
• reading another version of the changed metadata record as of the previous backup, the other version thereby being a previous version of the changed metadata record;
• comparing the current and previous versions of the changed metadata records to detect events that occurred between the previous and current backups;
• upon detecting that an event of a first type has occurred, recording, in a journal, information about the event of the first type, wherein the journal is separate from the changed block list generated by the CBT driver;
• and
• upon detecting that an event of a second type has occurred, not recording, in the journal, information about the event of the second type, wherein the event of the second type comprises a creation of a file after the previous backup and a deletion of the file before the current backup, wherein the detecting that the event of a first type has occurred further comprises:
• reading an entry for a file object in the previous version of the changed metadata record, wherein an in-use flag field in the previous version of the changed metadata record comprises a value indicating that the previous version of the changed metadata record was in-use as of the previous backup;
• reading an entry for a file object in the current version of the changed metadata record, wherein an in-use flag field in the current version of the changed metadata record comprises a value indicating that the current version of the changed metadata record was in-use as of the current backup;
• comparing a generation counter value stored in a generation counter field of the previous version of the changed metadata record and a generation counter value stored in a generation counter field of the current version of the changed metadata record to determine whether or not the generation counter values match;
• if the generation counter values match, determining that the file object associated with the current version of the changed metadata record has been modified and recording, in the journal, information about the file object having been modified;
• and
• if the generation counter values do not match, determining that the file object associated with the previous version of the changed metadata record has been deleted and recording, in the journal, information about the file object having been deleted.

A computer program product, comprising a non-transitory computer-readable medium having a computer-readable program code embodied therein, the computer-readable program code adapted to be executed by one or more processors to implement a method for indexing contents of an incremental block-based backup of a volume having a file system, the method comprising:

• allowing the incremental block-based backup of the volume to complete, the incremental backup being a current backup and comprising storing on backup storage all blocks of the volume that have changed since a previous backup according to a changed block list generated by a change block tracking (CBT) driver that intercepts writes from the file system;
• after the incremental backup has completed,
• obtaining the changed block list, the changed block list being a first list;
• reading superblocks of the file system to generate a second list identifying metadata blocks at which metadata records associated with files of the file system are stored;
• intersecting the first and second lists to return a third list identifying changed metadata blocks, the changed metadata blocks thereby corresponding to metadata records that have changed between the previous and current backups;
• reading a changed metadata record, the changed metadata record being a current version of the changed metadata record;
• reading another version of the changed metadata record as of the previous backup, the other version thereby being a previous version of the changed metadata record;
• comparing the current and previous versions of the changed metadata records to detect events that occurred between the previous and current backups;
• upon detecting that an event of a first type has occurred, recording, in a journal, information about the event of the first type, wherein the journal is stored separate from the changed block list;
• and
• upon detecting that an event of a second type has occurred, not recording, in the journal, information about the event of the second type, wherein the event of the second type comprises a creation of a file after the previous backup and a deletion of the file before the current backup, wherein the detecting that the event of a first type has occurred further comprises:
• reading an entry for a file object in the previous version of the changed metadata record, wherein an in-use flag field in the previous version of the changed metadata record comprises a value indicating that the previous version of the changed metadata record was in-use as of the previous backup;
• reading an entry for a file object in the current version of the changed metadata record, wherein an in-use flag field in the current version of the changed metadata record comprises a value indicating that the current version of the changed metadata record was in-use as of the current backup;
• comparing a generation counter value stored in a generation counter field of the previous version of the changed metadata record and a generation counter value stored in a generation counter field of the current version of the changed metadata record to determine whether or not the generation counter values match;
• if the generation counter values match, determining that the file object associated with the current version of the changed metadata record has been modified and recording, in the journal, information about the file object having been modified;
• and
• if the generation counter values do not match, determining that the file object associated with the previous version of the changed metadata record has been deleted and recording, in the journal, information about the file object having been deleted.