Namespace management in a distributed file system

#The present invention generally relates to distributed file systems, and more particularly to management of a namespace in a distributed file system.

#A partition-based approach to achieve high scalability for access to distributed storage services is currently being explored. The partition-based approach addresses the inherent scalability problems of cluster file systems, which are due to contention for the globally shared resources. In a partition-based approach, the resources of the system are divided into partitions, with each partition stored on a different partition server. Shared access is controlled on a per-partition basis.

#All implementations of partition-based distributed storage services must maintain namespaces, which generally are distributed and reference objects that reside in multiple partitions. A namespace provides a mapping between names and physical objects in the system (e.g., files). A user usually refers to an object by a textual name. The textual name is mapped to a lower-level reference that identifies the actual object, including a location and object identifier. The namespace is implemented by means of directories, which are persistent files of <name, reference> pairs.

#Various aspects and advantages of the invention will become apparent upon review of the following detailed description and upon reference to the drawings in which:

#FIG. 1 is a functional block diagram of an example distributed file system that is implemented with partition servers;

#FIG. 2 is a block diagram that illustrates links from example namespace objects to an example target object and links from the target objects back to the namespace object;

#In various embodiments, the present invention provides a set of protocols for implementing the two main classes of operations (link and unlink) performed in distributed namespaces. The protocol minimizes the runtime overhead incurred in support of recoverability from host and communications failures by reducing the number of synchronous input/output operations in the critical path of the operation. In addition, the protocols avoid locking distributed resources by serializing operations at each partition server. In most failure scenarios, the protocols use aggressive recovery techniques to replay incomplete operations. Overall, the invention reduces communications overhead, reduces synchronous I/O and increases operation concurrency.

#As described above, the two main classes of namespace operations are link and unlink. Other namespace operations can be either reduced to or composed by these two primitives. Table 1 below illustrates mappings of example file service operations to the associated namespace operations.

#TABLE 1 File Service operation Namespace primitive(s) create/mkdir obtain a new object + link remove/rmdir/unlink unlink rename link (to_dir) + unlink(from_dir)