# US 2017/0255522 A1 · Data storage method, data recovery method, related apparatus, and system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0255522 A1
- Document ID: US20170255522A1
- Status: Published application
- Kind: A1
- Application number: 15/603,413
- Filed: 2017-05-23
- Published: 2017-09-07
- Earliest claimed: 2015-06-04
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co.,Ltd.
- Inventors: Daohui Wang; Feng Zhang; Wei Fan; Zhile Zhang; Yongqiang Zeng
- Web page: https://patentgrep.com/patent/US20170255522A1

## CPC

- G06F 11/1076: {Parity data used in redundant arrays of independent storages, e.g. in RAID systems}
- G06F 3/0617: {in relation to availability}
- G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors}
- G06F 3/0631: {by allocating resources to storage systems}
- G06F 3/064: {Management of blocks}
- G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}
- G06F 9/466: {Transaction processing}
- G06F 9/5011: {the resources being hardware resources other than CPUs, Servers and Terminals}
- G06F 11/00: Error detection; Error correction; Monitoring (error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A data recovery method to improve storage consistency of a distributed storage system includes: receiving a recovery message from a storage node after a fault recovery of the faulty storage node; obtaining N lost erasure coding (EC) blocks of the faulty storage node through computation according to EC blocks stored by non-faulty storage nodes; sequentially sending N lost EC blocks to the faulty storage node for storage. In the process of sequentially sending the N lost EC blocks to the faulty storage node, when it is determined that for a first EC block that is to be stored by the faulty storage node, a second EC block having a same key value as the first EC block exists in the lost EC blocks, after the faulty storage node stores the second EC block, the first EC block is sent to the faulty storage node to store.

## Claims

### Claim 1

A data recovery method for use in a distributed storage system, wherein a storage node group of the distributed storage system comprises at least m+k storage nodes, in which one storage node is a primary storage node and others are secondary storage nodes, wherein m and k are both positive integers, and wherein the distributed storage system stores data by using erasure coding (EC) stripes, each EC stripe comprises m data blocks and k parity blocks, each EC stripe corresponds to a key value, key values of EC stripes are the same when the key values of EC strips have same address ranges of data parts, and key values of EC stripes are different when EC strips have different address ranges of data parts,
wherein the data recovery method is performed by the primary storage node, when it is detected that one of the secondary storage nodes is faulty, and after the faulty storage node is recovered, the method comprises:
receiving a recovery message sent by the faulty storage node after fault recovery of the faulty storage node;
obtaining N lost EC blocks of the faulty storage node through computation according to EC blocks stored by non-faulty storage nodes in the storage node group, wherein N is a positive integer;
sequentially sending the N lost EC blocks to the faulty storage node for storage;
in the process of sequentially sending the N lost EC blocks to the faulty storage node, when the primary storage node generates a first EC block and determines that the first EC block is to be stored by the faulty storage node, determining whether a second EC block having a same key value as the first EC block exists in the lost EC blocks that are not stored successfully by the faulty storage node; and
when it is determined that the second EC block exists, after the faulty storage node successfully stores the second EC block, sending the first EC block to the faulty storage node to store.

### Claim 2 (depends on 1)

The data recovery method according to claim 1, wherein the sequentially sending the N lost EC blocks to the faulty storage node for storage comprises:
sending an nth lost EC block to the faulty storage node, wherein 1<n<N;
receiving a response message corresponding to the nth lost EC block, wherein the response message is sent by the faulty storage node, and the response message indicates that the faulty storage node has successfully stored the nth lost EC block; and
after receiving the response message of the faulty storage node with respect to the nth lost EC block, if n<N, incrementing n by 1, and re-executing the step of sending an nth lost EC block to the faulty storage node.

### Claim 3 (depends on 2)

The data recovery method according to claim 2, wherein the lost EC blocks that are not stored successfully comprise the lost EC blocks that are not sent to the faulty storage node, and the lost EC blocks that are sent to the faulty storage node but for which a response corresponding message is not received.

### Claim 4 (depends on 1)

The data recovery method according to claim 1, wherein a logical volume of the distributed storage system comprises multiple logical partitions, each logical partition has a size of k data blocks, and each logical partition does not overlap another, and wherein each logical partition uniquely corresponds to one key value, and a key value of a logical partition that each EC block falls within is a key value of the EC block.

### Claim 5

A data recovery apparatus in a distributed storage system, wherein a storage node group of the distributed storage system comprises at least m+k storage nodes, in which the data storage apparatus is a primary storage node and others are secondary storage nodes, wherein m and k are both positive integers, and wherein the distributed storage system stores data by using erasure coding (EC) stripes, each EC stripe comprises m data blocks and k parity blocks, each EC stripe corresponds to a key value, key values of EC stripes are the same when the key values of EC strips have same address ranges of data parts, and key values of EC stripes are different when EC strips have different address ranges of data parts,
wherein the data storage apparatus comprises:
an input/output interface,
a processor, and
a memory storing instructions for execution by the processor,
wherein the instructions, when executed by the processor, cause the data storage apparatus to perform a process that comprises:
receiving a recovery message sent by the faulty storage node after fault recovery of the faulty storage node;
obtaining N lost EC blocks of the faulty storage node through computation according to EC blocks stored by non-faulty storage nodes in the storage node group, wherein N is a positive integer;
sequentially sending the N lost EC blocks to the faulty storage node for storage;
in the process of sequentially sending the N lost EC blocks to the faulty storage node, when the primary storage node generates a first EC block and determines that the first EC block is to be stored by the faulty storage node, determining whether a second EC block having a same key value as the first EC block exists in the lost EC blocks that are not stored successfully by the faulty storage node; and
when it is determined that the second EC block exists, after the faulty storage node successfully stores the second EC block, sending the first EC block to the faulty storage node to store.

### Claim 6 (depends on 5)

The data recovery apparatus according to the claim 5, the process further comprises:
sending an nth lost EC block to the faulty storage node, wherein 1<n<N;
after receiving a response message of the faulty storage node with respect to the nth lost EC block, if n<N, incrementing n by 1, and re-execute the step of sending an nth lost EC block to the faulty storage node, wherein the response message indicates that the faulty storage node has successfully stored the nth lost EC block.

### Claim 7 (depends on 6)

The data recovery apparatus according to the claim 6, wherein the lost EC blocks that are not stored successfully comprise the lost EC blocks that are not sent to the faulty storage node and the lost EC blocks that are sent to the faulty storage node but for which a response corresponding message is not received.

### Claim 8 (depends on 5)

The data recovery apparatus according to the claim 5, wherein a logical volume of the distributed storage system comprises multiple logical partitions, each logical partition has a size of k data blocks, and each logical partition does not overlap another, and wherein each logical partition uniquely corresponds to one key value, and a key value of a logical partition that each EC block falls within is a key value of the EC block.
