Automatic load spreading in a clustered network storage system

A method comprising:

• maintaining a plurality of data storage nodes in a clustered network storage server system, each data storage node including an N-module and a D-module;
• maintaining, in the N-module of each data storage node, a data structure indicating a mapping between D-modules and data identifiers of data stored in the clustered network storage system;
• and
• spreading read and write accesses across the plurality of data storage nodes, including:
• receiving at one of the N-modules a write request directed to a first dataset;
• determining one of the D-modules at which to access the first dataset, based on the mapping and a data identifier indicated by the write request;
• directing the write request to said one of the D-modules;
• receiving at one of the N-modules a read request directed to a second dataset;
• selecting by said one of the N-module either a caching D-module for the second dataset or a read-write D-module for the second dataset, from which to access the second dataset, based on an indication whether the second dataset has been updated at the read-write D-module within a threshold time;
• and
• directing, by said one of the N-modules, the read request to either said caching D-module or said read-write D-module selected in said selecting.

A clustered network storage system comprising:

• a plurality of storage nodes configured to operate as a cluster, each of the data storage nodes including a mass storage subsystem and a D-module associated therewith to manage back-end storage functions for the mass storage subsystem, each of the D-modules to implement a cache for primary data stored in one or more of the mass storage subsystems on one or more of the storage nodes other than the mass storage subsystem on the storage node with which the D-module is associated;
• and
• a plurality of N-modules, each to receive data access requests via a network from one or more of a plurality of clients, such that each data access request received by any of the N-modules is mapped to a particular one of the D-modules for servicing so as to spread file system data and metadata access load across the plurality of data storage nodes, the plurality of N-modules configured to:
• receive a write request directed to a first dataset;
• determine one of the D-modules at which to access the first dataset, based on the mapping and a data identifier indicated by the write request;
• direct the write request to said determined one of the D-modules;
• receive a read request directed to a second dataset;
• select either a caching D-module for the first dataset or a read-write D-module for the first dataset, from which to access the first dataset, based on the mapping and an indication whether the first dataset has been updated at the read-write D-module within a threshold time;
• and
• direct the read request to either said selected caching D-module or read-write D-module.