Back to results
US 7,937,421 B2Granted
Systems and methods for restriping files in a distributed file system
AssigneeEMC Corporation
InventorsPaul A. Mikesell, Rob Anderson, Aaron James Passey, Peter John Godman, Hassan F. Khan, Darren P. Schack
Life of this patent
Estimating the term…
Abstract
Systems and methods for restriping files distributed among a set of smart storage units, wherein data blocks for a particular stripe do not typically need to be located at any particular location on the storage units, wherein data can be typically restriped among the smart storage units with minimal data movement, and wherein data is typically protected and recoverable even if a system failure occurs during the restriping process.
Claims13
A distributed file storage system comprising:
- multiple storage units configured to communicate with each other, each storage unit comprising a storage device, a processor, and at least one executable software module stored on the storage unit, the processor configured to execute the at least one executable software module;
- a first file stored on the distributed file system in a first set of a plurality of stripes, the first file comprising multiple data blocks, each of the first set of a plurality of stripes comprising a subset of the multiple data blocks, wherein at least a first one of the multiple data blocks is stored on a first one of the multiple storage units and at least a second one of the multiple data blocks is stored on a second one of the multiple storage units;
- error correction data related to the first file stored on the distributed file system wherein the error correction data is configured to be used in conjunction with available data blocks to reconstruct the first file;
- a first metadata block associated with the first file stored on a third one of the multiple storage units, the first metadata block comprising an address location of each of the multiple data blocks of a stripe stored on the distributed file system and at least one address location of error correction data stored on the distributed file system;
- a second file stored on the distributed file system in a second set of a plurality of stripes;
- a second metadata block associated with the second file stored on a fourth one of the multiple storage units which is different than the third one of the multiple storage units, wherein the second metadata block is different than the first metadata block;
- wherein the at least one executable software module of each storage unit is configured to:
- access one or more metadata blocks stored on one or more different storage units;
- restripe, according to one or more preferences, at least a portion of the first file among two or more of the multiple storage units, wherein the one or more preferences comprise one or more of the following:
- minimizing movement of data, distributing the at least a portion of the first file or at least a portion of the error correction data among two or more of an optimal or preferred subset of the multiple storage units, distributing the at least a portion of the first file or at least a portion of the error correction data among two or more of an optimal or preferred ordered subset of the multiple storage units, or distributing the at least a portion of the first file or at least a portion of the error correction data based on one or more performance metrics;
- restripe, according to the one or more preferences, at least a portion of the error correction data among two or more of the multiple storage units;
- and
- update the first metadata block to include a new address location for each of the multiple data blocks that was restriped and a new address location for each of the error correction data that was restriped, wherein the new address location includes data locations on different storage units.
10 more elements
A distributed file storage system comprising:
- multiple storage units configured to communicate with each other, each storage unit comprising a storage device, a processor, and at least one executable software module;
- a first data file stored on the distributed file system in multiple stripes, the first data file comprising multiple data blocks, each of the multiple stripes comprising a subset of the multiple data blocks, wherein the multiple data blocks are distributed among a subset of two or more said multiple storage units;
- error correction data related to the first data file comprising multiple error correction blocks, the multiple error correction blocks distributed among a subset of two or more said multiple storage units;
- a first metadata associated with the first data file, the first metadata stored on at least one of the multiple storage units and comprising locations of each of said multiple data blocks of each stripe of the first data file and locations of each of said multiple error correction blocks;
- and
- a second metadata associated with a second data file, wherein the second metadata is different than the first metadata;
- wherein the location of the first metadata is not dependent on the locations of each of said multiple data blocks, the locations of each of said multiple error correction blocks, or the location of the second metadata;
- and
- wherein said at least one executable software module of each storage unit is configured to:
- access metadata stored on a different storage unit;
- reconstruct at least a subset of said multiple data blocks using at least a subset of said multiple error correction blocks and a subset of said multiple data blocks;
- redistribute at least a subset of said multiple data blocks among a subset of two or more said multiple storage units based on one or more preferences, wherein the one or more preferences comprise one or more of the following:
- minimizing movement of data, distributing the at least a portion of the first file or at least a portion of the error correction data among two or more of an optimal or preferred subset of the multiple storage units, distributing the at least a portion of the first file or at least a portion of the error correction data among two or more of an optimal or preferred ordered subset of the multiple storage units, or distributing the at least a portion of the first file or at least a portion of the error correction data based on one or more performance metrics;
- redistribute at least a subset of said multiple error correction blocks among a subset of two or more said multiple storage units based on the one or more preferences;
- update the first metadata to include a new location for one or more of said multiple data blocks;
- and
- update the first metadata to include a new location for one or more of said multiple error correction blocks.
14 more elements
Dependencies are inferred from the claim text.