# US 2021/0342215 A1 · Generating recommendations for initiating recovery of a fault domain representing logical address space of a storage system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342215 A1
- Document ID: US20210342215A1
- Status: Published application
- Kind: A1
- Application number: 16/862,740
- Filed: 2020-04-30
- Published: 2021-11-04
- Earliest claimed: 2020-04-30
- Assignees: EMC IP Holding Company LLC
- Applicants: EMC IP Holding Company LLC
- Inventors: Rohit K. Chawla; Dixitkumar Vishnubhai Patel
- Web page: https://patentgrep.com/patent/US20210342215A1

## CPC

- G06F 11/0793: {Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40)}
- G06F 11/0727: {in a storage system, e.g. in a DASD or network based storage system (drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097)}
- G06F 11/0751: {Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06)}
- G06F 11/0754: {by exceeding limits}
- G06F 11/079: {Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36)}
- G06F 12/0238: {Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory}
- G06F 12/1009: using page tables, e.g. page table structures
- G06F 2212/1016: Performance improvement
- G06F 2212/1032: Reliability improvement, data loss prevention, degraded operation etc
- G06F 2212/1044: Space efficiency improvement
- G06F 2212/7201: Logical to physical mapping or translation of blocks or pages
- G06F 2212/7204: Capacity control, e.g. partitioning, end-of-life degradation
- G06F 2212/7208: Multiple device management, e.g. distributing data over multiple flash devices
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An apparatus comprises a processing device configured to identify faults associated with a logical address space in a fault domain of a storage system, the faults specifying fault reason codes and metadata types for logical pages in the logical address space associated with the faults. The processing device is also configured to determine a fault summary characterizing impact of the faults in the fault domain of the storage system, the fault summary being based on aggregating fault scores assigned to the fault reason codes and the metadata types specified in the faults. The processing device is further configured to generate a recommendation on whether to initiate recovery of the fault domain of the storage system based on the fault summary, and to initiate recovery of the fault domain of the storage system based on the generated recommendation.

## Claims

### Claim 1

An apparatus comprising:
at least one processing device comprising a processor coupled to a memory;
the at least one processing device being configured to perform steps of:
identifying a plurality of faults associated with at least a portion of a logical address space in a fault domain of a storage system, the plurality of faults specifying (i) fault reason codes and (ii) metadata types for logical pages in the logical address space associated with the plurality of faults;
determining a fault summary characterizing impact of the plurality of faults in the fault domain of the storage system, the fault summary being based at least in part on aggregating fault scores assigned to the fault reason codes and the metadata types specified in the plurality of faults;
generating a recommendation on whether to initiate recovery of the fault domain of the storage system based at least in part on the fault summary; and
initiating recovery of the fault domain of the storage system based at least in part on the generated recommendation.

### Claim 2 (depends on 1)

The apparatus of claim 1 wherein the fault domain comprises all of the logical address space of the storage system.

### Claim 3 (depends on 1)

The apparatus of claim 1 wherein the logical address space is organized as a B-tree comprising a plurality of levels, the plurality of levels comprising a leaf logical page level comprising a plurality of leaf pages and one or more additional logical page levels above the leaf logical page level.

### Claim 4 (depends on 3)

The apparatus of claim 3 wherein the one or more additional logical page levels comprise:
a middle page level comprising a plurality of middle pages associated with respective subsets of the plurality of leaf pages in the leaf page level; and
a top page level comprising one or more top pages associated with respective subsets of the plurality of middle pages in the middle page level.

### Claim 5 (depends on 4)

The apparatus of claim 4 wherein a given one of the one or more top pages represents an n*m sized portion of the logical address space that references n of the plurality of middle pages in the middle page level, a given one of the n middle pages represents an m sized portion of the logical address space and references n of the plurality of leaf pages in the leaf page level, and a given one of the n leaf pages represents an m/n sized portion of the logical address space.

### Claim 6 (depends on 5)

The apparatus of claim 5 wherein n is 512 and m is one gigabyte.

### Claim 7 (depends on 3)

The apparatus of claim 3 wherein a fault score assigned to a metadata type for the leaf logical page level is lower than a fault score assigned to metadata types for the one or more additional logical page levels above the leaf page logical level.

### Claim 8 (depends on 3)

The apparatus of claim 3 wherein the plurality of leaf pages in the leaf logical page level comprise pointers to virtual block addresses associated with entries in a plurality of virtual blocks in a virtual block level of the logical address space, wherein the virtual block addresses comprise pointers to physical block addresses in a plurality of physical blocks in a physical block level of the logical address space, wherein a fault score assigned to a metadata type for the virtual block level is lower than a fault score assigned to a metadata type for the leaf logical page level, and wherein a fault score assigned to a metadata type for the physical block level is lower than a fault score assigned to the metadata type for the virtual block level.

### Claim 9 (depends on 1)

The apparatus of claim 1 wherein a given one of the plurality of faults specifies:
a given metadata type for a given logical page in the logical address space that is a source of the given fault;
a given fault reason code;
a given fault scope characterizing at least one of potential data loss and potential logical address space loss in the storage system resulting from the given fault;
a given snapshot group associated with the given logical page; and
one or more storage volumes associated with the given logical page.

### Claim 10 (depends on 1)

The apparatus of claim 1 wherein the fault summary comprises a set of fault summary parameters for the plurality of faults, the set of fault summary parameters comprising:
at least one of a number of unique faults in the plurality of faults, a number of storage volumes impacted by the plurality of faults, and a number of snapshot groups impacted by the plurality of faults;
at least one of a total amount of data made unavailable in the storage system as a result of the plurality of faults, a total amount of logical address space made unavailable as a result of the plurality of faults, per-storage volume amounts of data made unavailable in the storage system as a result of the plurality of faults, and per-snapshot group amounts of data made unavailable in the storage system as a result of the plurality of faults; and
at least one of a total amount of recoverable data in the storage system, per-storage volume amounts of recoverable data in the storage system, and per-snapshot group amounts of recoverable data in the storage system.

### Claim 11 (depends on 1)

The apparatus of claim 1 wherein the fault summary comprises a set of fault summary parameters for the plurality of faults, the set of fault summary parameters comprising:
an average fault score for the metadata types specified in the plurality of faults;
a standard deviation of the average fault score for the metadata types specified in the plurality of faults;
an average fault score for the fault reason codes specified in the plurality of faults; and
a standard deviation of the average fault score for the fault reason codes specified in the plurality of faults.

### Claim 12 (depends on 11)

The apparatus of claim 11 wherein the fault summary comprises at least one visualization of the set of fault summary parameters, the at least one visualization comprising a plot comprising:
data points for each of at least a subset of the plurality of faults, a given data point for a given fault representing the fault score for the metadata type specified in the given fault on a first axis and the fault score for the fault reason code specified in the given fault on a second axis;
an additional data point representing the average fault score for the metadata types specified in the plurality of faults on the first axis and the average fault score for the fault reason codes specified in the plurality of faults on the second axis;
a first visual indicator of the standard deviation of the average fault score for the metadata types specified in the plurality of faults extending from the additional data point along the first axis;
a second visual indicator of the standard deviation of the average fault score for the fault reason codes specified in the plurality of faults extending from the additional data point along the second axis; and
a third visual indicator representing a radius of fault scores in the first axis and the second axis that result in generating a recommendation to initiate the recovery of the fault domain of the storage system.

### Claim 13 (depends on 11)

The apparatus of claim 11 wherein generating the recommendation on whether to initiate the recovery of the fault domain of the storage system comprises generating a recommendation to initiate the recovery responsive to at least one of:
the average fault score for the metadata types specified in the plurality of faults being above a first designated threshold and the standard deviation of the average fault score for the metadata types specified in the plurality of faults being below a second designated threshold; and
the average fault score for the fault reason codes specified in the plurality of faults being above a third designated threshold and the standard deviation of the average fault score for the fault reason codes specified in the plurality of faults being below a fourth designated threshold.

### Claim 14 (depends on 1)

The apparatus of claim 1 wherein the fault summary comprises a set of fault summary parameters for the plurality of faults, the set of fault summary parameters comprising:
a frequency distribution of fault scores for the metadata types specified in the plurality of faults; and
a frequency distribution of fault scores for the fault reason codes specified in the plurality of faults.

### Claim 15

A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by at least one processing device causes the at least one processing device to perform steps of:
identifying a plurality of faults associated with at least a portion of a logical address space in a fault domain of a storage system, the plurality of faults specifying (i) fault reason codes and (ii) metadata types for logical pages in the logical address space associated with the plurality of faults;
determining a fault summary characterizing impact of the plurality of faults in the fault domain of the storage system, the fault summary being based at least in part on aggregating fault scores assigned to the fault reason codes and the metadata types specified in the plurality of faults;
generating a recommendation on whether to initiate recovery of the fault domain of the storage system based at least in part on the fault summary; and
initiating recovery of the fault domain of the storage system based at least in part on the generated recommendation.

### Claim 16 (depends on 15)

The computer program product of claim 15 wherein the fault summary comprises a set of fault summary parameters for the plurality of faults, the set of fault summary parameters comprising:
an average fault score for the metadata types specified in the plurality of faults;
a standard deviation of the average fault score for the metadata types specified in the plurality of faults;
an average fault score for the fault reason codes specified in the plurality of faults; and
a standard deviation of the average fault score for the fault reason codes specified in the plurality of faults.

### Claim 17 (depends on 16)

The computer program product of claim 16 wherein the fault summary comprises at least one visualization of the set of fault summary parameters, the at least one visualization comprising a plot comprising:
data points for each of at least a subset of the plurality of faults, a given data point for a given fault representing the fault score for the metadata type specified in the given fault on a first axis and the fault score for the fault reason code specified in the given fault on a second axis;
an additional data point representing the average fault score for the metadata types specified in the plurality of faults on the first axis and the average fault score for the fault reason codes specified in the plurality of faults on the second axis;
a first visual indicator of the standard deviation of the average fault score for the metadata types specified in the plurality of faults extending from the additional data point along the first axis;
a second visual indicator of the standard deviation of the average fault score for the fault reason codes specified in the plurality of faults extending from the additional data point along the second axis; and
a third visual indicator representing a radius of fault scores in the first axis and the second axis that result in generating a recommendation to initiate the recovery of the fault domain of the storage system.

### Claim 18

A method comprising steps of:
identifying a plurality of faults associated with at least a portion of a logical address space in a fault domain of a storage system, the plurality of faults specifying (i) fault reason codes and (ii) metadata types for logical pages in the logical address space associated with the plurality of faults;
determining a fault summary characterizing impact of the plurality of faults in the fault domain of the storage system, the fault summary being based at least in part on aggregating fault scores assigned to the fault reason codes and the metadata types specified in the plurality of faults;
generating a recommendation on whether to initiate recovery of the fault domain of the storage system based at least in part on the fault summary; and
initiating recovery of the fault domain of the storage system based at least in part on the generated recommendation;
wherein the method is performed by at least one processing device comprising a processor coupled to a memory.

### Claim 19 (depends on 18)

The method of claim 18 wherein the fault summary comprises a set of fault summary parameters for the plurality of faults, the set of fault summary parameters comprising:
an average fault score for the metadata types specified in the plurality of faults;
a standard deviation of the average fault score for the metadata types specified in the plurality of faults;
an average fault score for the fault reason codes specified in the plurality of faults; and
a standard deviation of the average fault score for the fault reason codes specified in the plurality of faults.

### Claim 20 (depends on 19)

The method of claim 19 wherein the fault summary comprises at least one visualization of the set of fault summary parameters, the at least one visualization comprising a plot comprising:
data points for each of at least a subset of the plurality of faults, a given data point for a given fault representing the fault score for the metadata type specified in the given fault on a first axis and the fault score for the fault reason code specified in the given fault on a second axis;
an additional data point representing the average fault score for the metadata types specified in the plurality of faults on the first axis and the average fault score for the fault reason codes specified in the plurality of faults on the second axis;
a first visual indicator of the standard deviation of the average fault score for the metadata types specified in the plurality of faults extending from the additional data point along the first axis;
a second visual indicator of the standard deviation of the average fault score for the fault reason codes specified in the plurality of faults extending from the additional data point along the second axis; and
a third visual indicator representing a radius of fault scores in the first axis and the second axis that result in generating a recommendation to initiate the recovery of the fault domain of the storage system.
