Systems and methods for providing a distributed file system utilizing metadata to track information about data stored throughout the system
AssigneeIsilon Systems, Inc.
InventorsSujal M. Patel, Paul A. Mikesell, Darren P. Schack
Life of this patent
Estimating the term…
Abstract
The intelligent distributed file system enables the storing of file data among a plurality of smart storage units which are accessed as a single file system. The intelligent distributed file system utilizes a metadata data structure to track and manage detailed information about each file, including, for example, the device and block locations of the file's data blocks, to permit different levels of replication and/or redundancy within a single file system, to facilitate the change of redundancy parameters, to provide high-level protection for metadata, to replicate and move data in real-time, and so forth.
Claims8
A distributed file storage system comprising:
- a plurality of storage units, comprising:
- a first storage unit comprising a first processing module;
- a second storage unit comprising a second processing module;
- and
- a third storage unit comprising a third processing module, the third storage unit different from the first storage unit and the second storage unit;
- wherein the first processing module, the second processing module, and the third processing module are configured to process read requests for files stored in a distributed manner across the plurality of storage units;
- a file stored in a distributed manner across the plurality of storage units, the file comprising:
- a first file portion stored in the first storage unit;
- and
- a second file portion stored in the second storage unit, wherein the second file portion is different from the first file portion;
- and
- file metadata comprising identifications of storage units in which the first file portion and the second file portion are stored and a reference to a parity map, the parity map comprising identifications of storage units in which parity blocks for the file are stored, at least a first portion of the file metadata stored in the third storage unit;
- each of the first processing module, the second processing module, and the third processing module further configured to, in real-time:
- receive the first portion of file metadata stored in the third storage unit;
- receive the first file portion stored in the first storage unit and the second file portion stored in the second storage unit;
- generate a parity block based on at least the first file portion and the second file portion;
- select a fourth storage unit from the plurality of storage units on which to store the parity block based on one or more of a storage capacity, processor utilization and network utilization of one or more of the plurality of storage units, the fourth storage unit different from the first storage unit and the second storage unit to permit data recovery if the first storage unit or the second storage unit is unavailable;
- store the parity block on the fourth storage unit;
- and
- update the parity map to comprise an identification of the fourth storage unit.
Dependencies are inferred from the claim text.