# US 2014/0195847 A1 · Methods and systems of managing a distributed replica based storage > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0195847 A1 - Document ID: US20140195847A1 - Status: Published application - Kind: A1 - Application number: 14/239,170 - Filed: 2012-08-15 - Published: 2014-07-10 - Earliest claimed: 2011-08-17 - Assignees: ScalelO LLC - Inventors: Erez Webman; Lior Bahat; Eran Borovik; Dvir Koren; Boaz Palgi - Web page: https://patentgrep.com/patent/US20140195847A1 ## CPC - G06F 11/2094: {Redundant storage or storage space (G06F11/2056 takes precedence)} - G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors} - G06F 3/065: {Replication mechanisms} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 11/1076: {Parity data used in redundant arrays of independent storages, e.g. in RAID systems} - G06F 11/1425: {by reconfiguration of node membership} - G06F 11/3442: {for planning or managing the needed capacity} - G06F 2201/81: Threshold - G06F 2201/815: Virtual - G06F 2211/104: Metadata, i.e. metadata associated with RAID systems with parity - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method of managing a distributed storage space. The method comprises mapping a plurality of replica sets to a plurality of storage managing modules installed in a plurality of computing units, each of the plurality of storage managing modules manages access of at least one storage consumer application to replica data of at least one replica of a replica set from the plurality of replica sets, the replica data is stored in at least one drive of a respective the computing unit, allocating at least one time based credit to at least one of each storage managing module and the replica data, iteratively renewing the time based credit as long a failure of at least one of the storage managing module, and the at least one drive and the replica data is not detected plurality of storage managing. ## Claims ### Claim 1 A method of managing a distributed storage space, comprising; mapping a plurality of replica sets to a plurality of storage managing modules installed in a plurality of computing units, each of the plurality of storage managing modules manages access of at least one storage consumer application to replica data of at least one replica of a replica set from the plurality of replica sets, the replica data is stored in at least one drive of a respective the computing unit; allocating at least one time based credit to at least one of each storage managing module, the at least one drive and the replica data; iteratively renewing the time based credit as long a failure of at least one of the storage managing module, the at least one drive and the replica data is not detected. ### Claim 2 (depends on 1) The method of claim 1, further comprising reallocating the replica data to at least one other of the plurality of storage managing modules when the at least one time based credit is not renewed. ### Claim 3 (depends on 1) The method of claim 1, further comprising instructing a respective the storage managing module to reject access of the at least one storage consumer application to the at least one replica. ### Claim 4 (depends on 1) The method of claim 1, further comprising detecting a responsiveness of a respective the storage managing module and determining whether to reallocate the at least one replica to the storage managing module accordingly. ### Claim 5 (depends on 1) The method of claim 1, wherein the plurality of replica sets are part of a volume stored in a plurality of drives managed by the plurality of storage managing modules. ### Claim 6 (depends on 1) The method of claim 1, wherein each replica is divided to be stored in a plurality of volume allocation extents (VAEs) each define a range of consecutive addresses which comprise a physical segment in a virtual disk stored in the at least one drive. ### Claim 7 (depends on 1) The method of claim 1, wherein each of a plurality of volume allocation extents (VAEs) of each of the plurality of replicas is divided to be stored in a plurality of physical segments each of another of a plurality of virtual disks which are managed by the plurality of storage managing modules so that access to different areas of each VAE is managed by different storage managing modules of the plurality of storage managing modules. ### Claim 8 (depends on 1) The method of claim 1, wherein the plurality of computing units comprises a plurality of client terminals selected from a group consisting of desktops, laptops, tablets, and Smartphones. ### Claim 9 (depends on 1) The method of claim 1, wherein each storage managing module manages a direct access of the at least one storage consumer application to a respective the at least one replica. ### Claim 10 (depends on 1) The method of claim 1, wherein the mapping comprises allocating a first generation numerator to mapping element mapping the storage of the replica data, the reallocating comprises updating the first generation numerator; further comprising receiving a request to access the replica data with a second generation numerator and validating the replica data according to a match between the first generation numerator and the second generation numerator. ### Claim 11 (depends on 1) A computer readable medium comprising computer executable instructions adapted to perform the method of claim 1. ### Claim 12 (depends on 1) The method of claim 1, further comprising performing a liveness check to the plurality of storage managing modules and performing the renewing based on an outcome of the liveness check. ### Claim 13 (depends on 1) The method of claim 1, wherein the replica set is defined according to a member of a group consisting of the following protocols: Redundant Array of Independent Disks (RAID)-0 protocol, RAID-1, RAID-2, RAID-3, RAID-4, RAID-5 and RAID-6, RAID 10, RAID 20, RAID 30, RAID 40, RAID 50, RAID 60, RAID 01, RAID 02, RAID 03, RAID 04, RAID 05, and RAID 06; wherein the replica comprises at least one of a replica of data of a set of data elements and a parity of the set of data elements. ### Claim 14 A system of managing a distributed storage space, comprising; a plurality of storage managing modules installed in a plurality of computing units and manages the storage of a plurality of replica sets, each storage managing module manages access of at least one storage consumer application to replica data of at least one replica of a replica set from the plurality of replica sets, the replica data is stored in at least one drive of a respective the computing unit; and a central node which allocates at least one time based credit to at least one of each storage managing module and the replica data; wherein the central node iteratively renews the time based credit as long a failure of at least one of the storage managing module, the at least one drive and the replica data is not detected. ### Claim 15 (depends on 14) The system of claim 14, wherein the central node reallocates the replica data to at least one other of the plurality of storage managing modules when the at least one time based credit is not renewed. ### Claim 16 A method of managing a data-migration operation, comprising: using a first storage managing module of a plurality of storage managing modules to manage access of a plurality of storage consumer applications to a plurality of data blocks of data stored in at least one drive; identifying a failure of at least one of the first storage managing module and the at least one drive; initializing a rebuild operation of the data by forwarding of the plurality of data blocks to be managed by at least one other of the plurality of storage managing modules in response to the failure; identifying, during the rebuild operation, a recovery of at least one of the first storage managing module and the at least one drive; and determining per each of the plurality of data blocks which has been or being forwarded, whether to update a respective the data block according to changes to another copy thereof or to map the respective data block to be managed by the at least one other storage managing module based on a scope of the changes. ### Claim 17 (depends on 16) The method of claim 16, further comprising limiting a number of data blocks which are concurrently forwarding during the rebuild operation. ### Claim 18 (depends on 16) The method of claim 16, wherein the identifying a failure is performed after a waiting period has elapsed. ### Claim 19 (depends on 16) The method of claim 16, further comprising performing at least one of the rebuild operation according to the determining and rebalancing the plurality of storage managing modules according to the outcome of the rebuild operation. ### Claim 20 (depends on 19) The method of claim 19, wherein the rebalancing is performed according to a current capacity of each plurality of storage managing modules. ### Claim 21 (depends on 16) The method of claim 16, wherein the determining comprises identifying the changes in at least one virtual disk in a copy of the plurality of data blocks of the at least one other storage managing module. ### Claim 22 (depends on 16) A computer readable medium comprising computer executable instructions adapted to perform the method of claim 16. ### Claim 23 A system of managing a data-migration operation, comprising: a plurality of storage managing modules each manages access of a plurality of storage consumer applications to a plurality of data blocks of data stored in at least one drive; and a central node which identifies a failure of a first of the plurality of storage managing modules; wherein the central node initializes a rebuild operation of the data by instructing the forwarding of the plurality of data blocks to be managed by at least one other of the plurality of storage managing modules in response to the failure, identifies, during the rebuild operation, a recovery of at least one of the first storage managing module and the at least one drive, and determines per each of the plurality of data blocks which has been or being forwarded to the at least one storage managing module, whether to acquire changes thereto or to map the respective data block to be managed by the at least one other storage managing module based on a scope of the changes. ### Claim 24 A method of managing a distributed storage space, comprising; mapping a plurality of replica sets to a storage space managed by a plurality of storage managing modules installed in a plurality of computing units, each of the plurality of storage managing modules manages access of at least one storage consumer application to replica data of at least one replica of a replica set from the plurality of replica sets, the replica data is stored in at least one drive of a respective the computing unit, monitoring a storage capacity managed by each of the plurality of storage managing modules while the plurality of storage managing modules manage access of the at least one storage consumer application to the replica set, detecting an event which changes a mapping of the storage space to the plurality of storage managing modules, and rebalancing the storage space in response to the event by forwarding at least some of the replica data managed by a certain of the plurality of storage managing modules to at least one other storage managing module of the plurality of storage managing modules; Optionally, the event comprises an addition of at least one new storage managing module to the plurality of storage managing modules, the rebalancing comprises forwarding at least some of the replica data to the at least one new storage managing module. ### Claim 25 (depends on 24) The method of claim 24, wherein the event comprises an initiated removal of at least one of the plurality of storage managing modules. ### Claim 26 (depends on 24) The method of claim 24, wherein the event comprises a change in a respective the storage capacity of at least one of the plurality of storage managing modules. ### Claim 27 (depends on 24) The method of claim 24, wherein the rebalancing comprises detecting a failure in one of the plurality of storage managing modules during the rebalancing and scheduling at least one rebalancing operation pertaining to the rebalancing according to at least one data forwarding operation pertaining to a recovery of the failure. ### Claim 28 (depends on 24) The method of claim 24, wherein the replica set is stored in a plurality of virtual disks (VDs) which are managed by the plurality of storage managing modules, the rebalancing is performed by forwarding a group of the plurality of virtual disks among the plurality of storage managing modules.