Read tail latency reduction

A method of reading data in a data storage system including a plurality of Data Storage Devices (DSDs) storing a plurality of erasure coded shards, each DSD of the plurality of DSDs storing one or more erasure coded shards of the plurality of erasure coded shards, wherein the plurality of erasure coded shards is generated from an erasure coding on initial data, the method comprising:

• determining an individual latency indicator for each DSD of the plurality of DSDs based at least in part on a respective monitored time to complete at least one command for each DSD, wherein each individual latency indicator corresponds to a latency in retrieving an erasure coded shard stored in a respective DSD;
• determining a plurality of collective latency indicators based at least in part on the determined individual latency indicators, wherein each of the plurality of collective latency indicators corresponds to a latency in retrieving multiple erasure coded shards of the plurality of erasure coded shards from a respective particular subset of the plurality of DSDs;
• computing a respective probability that each collective latency indicator would exceed a latency limit;
• and
• selecting, based on the respective probabilities, a subset of the plurality of erasure coded shards to retrieve from at least a corresponding particular subset of the plurality of DSDs, to reduce read tail latency delay.

A Data Storage Device (DSD), comprising:

• a plurality of memory portions for storing a plurality of erasure coded shards, each memory portion of the plurality of memory portions for storing one or more erasure coded shards of the plurality of erasure coded shards, wherein the plurality of erasure coded shards is generated from an erasure coding on initial data;
• and
• a controller configured to:
• determine an individual latency indicator for each memory portion of the plurality of memory portions based at least in part on a respective monitored time to complete at least one command for each memory portion, wherein each individual latency indicator corresponds to a latency in retrieving an erasure coded shard stored in a respective memory portion;
• determine a plurality of collective latency indicators based at least in part on the determined individual latency indicators, wherein each of the plurality of collective latency indicators corresponds to a latency in retrieving multiple erasure coded shards of the plurality of erasure coded shards from a respective particular subset of the plurality of memory portions;
• compute a respective probability that each collective latency indicator would exceed a latency limit;
• and
• reduce read tail latency delay by selecting, based on the respective probabilities, a subset of the plurality of erasure coded shards to retrieve from at least a corresponding particular subset of the plurality of memory portions.

A host, comprising:

• an interface for communicating with a data storage system including a plurality of Data Storage Devices (DSDs) storing a plurality of erasure coded shards, each DSD of the plurality of DSDs storing one or more erasure coded shards of the plurality of erasure coded shards, wherein the plurality of erasure coded shards is generated from an erasure coding on initial data;
• and
• a processor configured to:
• determine an individual latency indicator for each DSD of the plurality of DSDs based at least in part on a respective monitored time to complete at least one command for each DSD, wherein each individual latency indicator corresponds to a latency in retrieving an erasure coded shard stored in a respective DSD;
• determine a plurality of collective latency indicators based at least in part on the determined individual latency indicators, wherein each of the plurality of collective latency indicators corresponds to a latency in retrieving multiple erasure coded shards of the plurality of erasure coded shards from a respective particular subset of the plurality of DSDs;
• compute a respective probability that each collective latency indicator would exceed a latency limit;
• and
• reduce read tail latency delay by selecting, based on the respective probabilities, a subset of the plurality of erasure coded shards to retrieve from at least a corresponding particular subset of the plurality of DSDs.

A Data Storage Device (DSD) for use in a data storage system including a plurality of DSDs storing a plurality of erasure coded shards, each DSD of the plurality of DSDs storing one or more erasure coded shards of the plurality of erasure coded shards, wherein the plurality of erasure coded shards is generated from an erasure coding on initial data, the DSD comprising:

• an interface for communicating with the plurality of DSDs;
• and
• a controller configured to:
• determine an individual latency indicator for each DSD of the plurality of DSDs based at least in part on a respective monitored time to complete at least one command for each DSD, wherein each individual latency indicator corresponds to a latency in retrieving an erasure coded shard stored in a respective DSD;
• determine a plurality of collective latency indicators based at least in part on the determined individual latency indicators, wherein each of the plurality of collective latency indicators corresponds to a latency in retrieving multiple erasure coded shards of the plurality of erasure coded shards from a respective particular subset of the plurality of DSDs;
• compute a respective probability that each collective latency indicator would exceed a latency limit;
• and
• reduce read tail latency delay by selecting, based on the respective probabilities, a subset of the plurality of erasure coded shards to retrieve from at least a corresponding particular subset of the plurality of DSDs.

A Data Storage Device (DSD) comprising:

• a plurality of memory portions storing a plurality of erasure coded shards, each memory portion of the plurality of memory portions storing one or more erasure coded shards of the plurality of erasure coded shards, wherein the plurality of erasure coded shards is generated from an erasure coding on initial data;
• means for determining an individual latency indicator for each memory portion of the plurality of memory portions based at least in part on a respective monitored time to complete at least one command for each memory portion, wherein each individual latency indicator corresponds to a latency in retrieving an erasure coded shard stored in a respective memory portion;
• means for determining a plurality of collective latency indicators based at least in part on the determined individual latency indicators, wherein each of the plurality of collective latency indicators corresponds to a latency in retrieving multiple erasure coded shards of the plurality of erasure coded shards from a respective particular subset of the plurality of memory portions;
• means for computing a respective probability that each collective latency indicator would exceed a latency limit;
• and
• means for selecting, based on the respective probabilities, a subset of the plurality of erasure coded shards to retrieve from at least a corresponding particular subset of the plurality of memory portions, to reduce read tail latency delay.