Snapshot and restoration of distributed file system

A data management system to process snapshots of a distributed file system, comprising at least one processor configured to perform operations including:

• storing file-to-chunk mapping in file system metadata, wherein the file-to-chunk mapping maps at least a file of the distributed file system to a respective set of data chunks that are included in the file at a first time associated with a first file system generation identifier (ID), and wherein each file of the distributed file system comprises multiple data chunks;
• creating, for each data chunk included in the distributed file system, a chunk generation ID by associating each data chunk with a respective file system generation ID, wherein the respective file system generation ID for a data chunk corresponds to a time at which an earliest snapshot that includes the data chunk was obtained;
• taking, at a second time associated with a second file system generation ID, a next-generation snapshot of the distributed file system, the next-generation snapshot comprising one or more data chunks of the distributed file system;
• incrementing, for all data chunks of the one or more data chunks included in the next-generation snapshot, the respective chunk generation IDs based at least in part on taking the next-generation snapshot;
• and
• updating, based at least in part on taking the next-generation snapshot of the distributed file system, the file-to-chunk mapping in the file system metadata, wherein the updated file-to-chunk mapping maps the file of the distributed file system to a second set of data chunks that are included in the file at the second time associated with the second file system generation ID.

A tangible machine-readable medium including instructions which, when read by a machine, cause a machine to perform operations in a method of processing snapshots in a distributed file system, the operations including:

• storing file-to-chunk mapping in file system metadata, wherein the file-to-chunk mapping maps at least a file of the distributed file system to a respective set of data chunks that are included in the file at a first time associated with a first file system generation identifier (ID), and wherein each file of the distributed file system comprises multiple data chunks;
• creating, for each data chunk included in the distributed file system, a chunk generation ID by associating each data chunk with a respective file system generation ID, wherein the respective file system generation ID for a data chunk corresponds to a time at which an earliest snapshot that includes the data chunk was obtained;
• taking, at a second time associated with a second file system generation ID, a next-generation snapshot of the distributed file system, the next-generation snapshot comprising one or more data chunks of the distributed file system;
• incrementing, for all data chunks of the one or more data chunks included in the next-generation snapshot, the respective chunk generation IDs based at least in part on taking the next-generation snapshot;
• and
• updating, based at least in part on taking the next-generation snapshot of the distributed file system, the file-to-chunk mapping in the file system metadata, wherein the updated file-to-chunk mapping maps the file of the distributed file system to a second set of data chunks that are included in the file at the second time associated with the second file system generation ID.