Garbage-collection in Log-based Block Devices with Snapshots

A system comprising:

• data processing hardware;
• and
• memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
• obtaining a request to compact a plurality of log files of a log-structured volume, the log-structured volume comprising a plurality of snapshots, each snapshot of the plurality of snapshots representative of a state of the log-structured volume at a different point in time;
• for each respective snapshot of the plurality of snapshots of the log-structured volume, determining fresh block runs of the plurality of log files used by the respective snapshot;
• for each respective log file of the plurality of log files:
• writing the fresh block runs of the respective log file to one or more respective compacted log files;
• and
• generating a respective per-log diff file mapping a location of the written fresh block runs at the respective log file to a location of the written fresh block runs at the one or more respective compacted log files;
• for each respective snapshot of the plurality of snapshots of the log-structured volume, generating a respective checkpoint based on respective per-log diff files;
• and
• deleting each respective log file of the plurality of log files.