Distributed file system that provides scalability and resiliency

A distributed storage system comprising:

• a plurality of nodes that form a cluster, wherein the cluster includes one or more central processing unit (CPU) resources and instructions that when executed by the one or more CPU resources cause the distributed storage system to:
• host a file system volume on a node of the plurality of nodes;
• expose the file system volume to a client of the cluster by an instance of a data management subsystem that provides file and data management functions;
• map file system data and file system metadata of the file system volume to a logical block address of a data block of a logical block device of an instance of a storage management subsystem that provides storage and block management functions, wherein the storage management subsystem is disaggregated from the data management subsystem;
• map the logical block address to a block identifier containing information indicative of a block store within the cluster and a location within the block store where the data block resides;
• store the block identifier as a key and the data block as a corresponding value in a key-value store;
• and
• scale either or both (i) a number of instances of the data management subsystem in accordance with demands of the client independently of a number of instances of the storage management subsystem and (ii) the number of instances of the storage management subsystem in accordance with storage demands on the cluster independently of the number of instances of the data management subsystem.

A non-transitory machine readable medium storing instructions, which when executed by one or more central processing unit (CPU) resources of a distributed storage system, cause the distributed storage system to:

• host a file system volume on a node of a plurality of nodes of a storage cluster;
• expose the file system volume to a client of the storage cluster by an instance of a data management subsystem that provides file and data management functions;
• map file system data and file system metadata of the file system volume to a logical block address of a data block of a logical block device of an instance of a storage management subsystem that provides storage and block management functions, wherein the storage management subsystem is disaggregated from the data management subsystem;
• map the logical block address to a block identifier containing information indicative of a block store within the storage cluster and a location within the block store where the data block resides;
• store the block identifier as a key and the data block as a corresponding value in a key-value store;
• and
• scale either or both (i) a number of instances of the data management subsystem in accordance with demands of the client independently of a number of instances of the storage management subsystem and (ii) the number of instances of the storage management subsystem in accordance with storage demands on the storage cluster independently of the number of instances of the data management subsystem.