Back to results
US 10,241,872 B2Granted
Hybrid distributed storage system
AssigneeAmplidata NV
InventorsFrederik De Schrijver, Bastiaan Stougie, Koen De Keyser, Wim De Wispelaere
Life of this patent
Estimating the term…
Abstract
There is provided a distributed object storage system that includes several performance optimizations with respect to efficiently storing data objects when coping with a desired concurrent failure tolerance of concurrent failures of storage elements which is greater than two and with respect to optimizing encoding/decoding overhead and the number of input and output operations at the level of the storage elements.
Claims21
A distributed object storage system comprising:
- a plurality of storage nodes, wherein:
- each storage node comprises a share of a plurality of storage elements of the distributed object storage system;
- the plurality of storage elements is adapted to redundantly store and retrieve a data object on a storage set;
- and
- the storage set comprises two or more storage elements of the plurality of storage elements;
- and
- at least one controller node coupled to or at least partly comprised within the plurality of storage nodes, the at least one controller node including a spreading module configured to:
- determine a desired concurrent failure tolerance of concurrent failures of storage elements of the storage set;
- select a level-1 fragment storage subset comprising a fragment spreading width of the storage elements of the storage set, the fragment spreading width being a sum of:
- a basic level-1 fragment storage element count corresponding to a number of storage elements of the level-1 fragment storage subset which are not allowed to fail, and
- a redundant level-1 fragment storage element count corresponding to a number of storage elements of the level-1 fragment storage subset which are allowed to concurrently fail;
- select a level-2 fragment storage subset comprising a level-2 fragment storage element count, which is equal to or greater than one, of the storage elements of the storage set, wherein:
- a sum of the redundant level-1 fragment storage element count and the level-2 fragment storage element count is equal to or greater than the desired concurrent failure tolerance,
- the basic level-1 fragment storage element count exceeds the level-2 fragment storage element count, and
- the data object is decodable from the level-2 fragment storage subset;
- store, on each storage element of the level-1 fragment storage subset, a level-1 fragment sub-collection comprising at least a level-1 encoding multiple of level-1 fragments generated by a hybrid encoding module;
- and
- store, on each storage element of the level-2 fragment storage subset, a level-2 fragment sub-collection comprising at least a level-2 encoding multiple of level-2 fragments generated by the hybrid encoding module;
- wherein the hybrid encoding module is configured to:
- generate a level-1 fragment collection comprising at least the level-1 encoding multiple multiplied by the fragment spreading width of level-1 fragments of the data object;
- and
- generate a level-2 fragment collection comprising at least the level-2 encoding multiple multiplied by the level-2 fragment storage element count of level-2 fragments of the data object;
- and
- wherein the at least one controller node is configured to determine a basic fragment count of one or more of level-1 fragments and level-2 fragments from one or more of the level-1 fragment storage subset and the level-2 fragment storage subset from which the data object is decodable.
21 more elements
A method of operating a distributed storage system, the method comprising:
- determining a desired concurrent failure tolerance of concurrent failures of storage elements of a storage set;
- selecting, by a spreading module, a level-1 fragment storage subset comprising a fragment spreading width of the storage elements of the storage set, the fragment spreading width being a sum of:
- a basic level-1 fragment storage element count corresponding to a number of storage elements of the level-1 fragment storage subset which are not allowed to fail, and
- a redundant level-1 fragment storage element count corresponding to a number of storage elements of the level-1 fragment storage subset which are allowed to concurrently fail;
- selecting, by the spreading module, a level-2 fragment storage subset comprising a level-2 fragment storage element count, which is equal to or greater than one, of the storage elements of the storage set, whereby a sum of the redundant level-1 fragment storage element count and the level-2 fragment storage element count is equal to or greater than the desired concurrent failure tolerance, wherein the basic level-1 fragment storage element count exceeds the level-2 fragment storage element count, and wherein a data object is decodable from the level-2 fragment storage subset;
- determining a basic fragment count of one or more of level-1 fragments and level-2 fragments stored by the spreading module from one or more of the level-1 fragment storage subset and the level-2 fragment storage subset from which the data object is decodable;
- generating, by a hybrid encoding module, a level-1 fragment collection comprising at least a level-1 encoding multiple multiplied by the fragment spreading width of level-1 fragments of the data object;
- and a level-2 fragment collection comprising at least a level-2 encoding multiple multiplied by the level-2 fragment storage element count of level-2 fragments of the data object;
- storing, on each storage element of the level-1 fragment storage subset, a level-1 fragment sub-collection comprising at least the level-1 encoding multiple of level-1 fragments generated by the hybrid encoding module;
- and
- storing, on each storage element of the level-2 fragment storage subset, a level-2 fragment sub-collection comprising at least the level-2 encoding multiple of level-2 fragments generated by the hybrid encoding module.
8 more elements
A distributed object storage system comprising:
- a plurality of storage nodes, wherein:
- each storage node comprises a share of a plurality of storage elements of the distributed object storage system;
- the plurality of storage elements is adapted to redundantly store and retrieve a data object on a storage set;
- and
- the storage set comprises two or more storage elements of the plurality of storage elements;
- means for determining a desired concurrent failure tolerance of concurrent failures of storage elements from the two or more storage elements of the storage set;
- means for selecting a level-1 fragment storage subset comprising a fragment spreading width of the storage elements of the storage set, the fragment spreading width being a sum of:
- a basic level-1 fragment storage element count corresponding to a number of storage elements of the level-1 fragment storage subset which are not allowed to fail;
- and
- a redundant level-1 fragment storage element count corresponding to a number of storage elements of the level-1 fragment storage subset which are allowed to concurrently fail;
- means for selecting a level-2 fragment storage subset comprising a level-2 fragment storage element count, which is equal to or greater than one, of the storage elements of the storage set, wherein:
- a sum of the redundant level-1 fragment storage element count and the level-2 fragment storage element count is equal to or greater than the desired concurrent failure tolerance;
- the basic level-1 fragment storage element count exceeds the level-2 fragment storage element count;
- and
- the data object is decodable from the level-2 fragment storage subset;
- means for storing, on each storage element of the level-1 fragment storage subset, a level-1 fragment sub-collection comprising at least a level-1 encoding multiple of level-1 fragments generated by a hybrid encoding module;
- means for storing, on each storage element of the level-2 fragment storage subset, a level-2 fragment sub-collection comprising at least a level-2 encoding multiple of level-2 fragments generated by the hybrid encoding module;
- means for generating a level-1 fragment collection comprising at least the level-1 encoding multiple multiplied by the fragment spreading width of level-1 fragments of the data object;
- means for generating a level-2 fragment collection comprising at least the level-2 encoding multiple multiplied by the level-2 fragment storage element count of level-2 fragments of the data object;
- and
- means for determining a basic fragment count of one or more of level-1 fragments and level-2 fragments from one or more of the level-1 fragment storage subset and the level-2 fragment storage subset from which the data object is decodable.
18 more elements
Dependencies are inferred from the claim text.