# US 11,030,038 B2 · Fault prediction and detection using time-based distributed data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,030,038 B2 - Document ID: US11030038B2 - Status: Granted - Kind: B2 - Application number: 16/416,107 - Filed: 2019-05-17 - Published: 2020-08-13 - Granted: 2021-06-08 - Earliest claimed: 2019-02-12 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Cosmos Darwin; Bryan S. Matthew; Donald MacGregor; Scott Chao-Chueh Lee - Web page: https://patentgrep.com/patent/US11030038B2 ## CPC - G06F 11/008: {Reliability or availability analysis} - G06F 11/0793: {Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40)} - G06F 3/061: {Improving I/O performance} - G06F 3/0614: {Improving the reliability of storage systems} - G06F 3/0653: {Monitoring storage devices or systems} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 11/0727: {in a storage system, e.g. in a DASD or network based storage system (drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097)} - G06F 11/0754: {by exceeding limits} - G06F 11/0757: {by exceeding a time limit, i.e. time-out, e.g. watchdogs} - G06F 11/079: {Root cause analysis, i.e. error or fault diagnosis (in a hardware test environment G06F11/22; in a software test environment G06F11/36)} - G06F 11/323: {Visualisation of programs or trace data} - G06F 11/3419: {by assessing time} - G06F 11/3452: {Performance evaluation by statistical analysis} - G06F 11/3485: {for I/O devices} - G06F 11/3409: {for performance assessment} - G06F 2201/81: Threshold - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Performance data is collected for input/output operations executed at a storage device of a plurality of storage devices of a software-defined storage network. Based on the collected performance data, a time-based I/O performance profile for the storage device is determined. A characteristic time-based I/O performance profile is determined for a representative group of storage devices having common characteristics with the storage device and based on previously collected performance data for devices of the representative group. It is determined that the difference between the time-based I/O performance profile for the storage device and the characteristic time-based I/O performance profile exceeds a predetermined deviance threshold that is indicative of a probable failure of the storage device. An indication is generated that the storage device exceeded the predetermined deviance threshold. ## Claims ### Claim 1 A computer-implemented method for predicting a state of a storage device, the method comprising: collecting, by a computing device, read/write latency data for input/output operations executed at a storage device of a plurality of storage devices of a software-defined storage network; based on the collected read/write latency data, determining, by the computing device, a time-distributed read/write latency performance profile for the storage device; determining, by the computing device, a characteristic time-distributed read/write latency performance profile for a representative group of storage devices having common characteristics with the storage device and based on previously collected performance data for devices of the representative group; determining, by the computing device, that a difference between the time-distributed read/write latency performance profile for the storage device and the characteristic time-distributed read/write latency performance profile exceeds a predetermined deviance threshold that is indicative of a probable failure of the storage device; and based on the determining that the storage device exceeded the predetermined deviance threshold, redirecting read/write requests for the storage device to other storage devices. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the plurality of storage devices are configured to implement a fault resiliency function. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the plurality of storage devices is an erasure coding group or a group of mirrored devices. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the collected performance data comprises latencies for I/O operations that were executed at the storage device. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the time-distributed read/write latency performance profile comprises a distribution of numbers of metrics that fell within each of a plurality of metric ranges. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the time-distributed read/write latency performance profile comprises a total number of I/O operations where associated latencies were greater than a threshold. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the time-distributed read/write latency performance profile for the storage device comprises a latency profile that is determined based on a number of latencies for the storage device that fall within each of a plurality of latency ranges and within a selected time period. ### Claim 8 A computing device comprising: one or more processors; a memory in communication with the one or more processors, the memory having computer-readable instructions stored thereupon which, when executed by the one or more processors, cause the computing device to perform operations comprising: determining a time-distributed read/write latency performance profile for a storage device, the time-distributed read/write latency performance profile determined based on one or more read/write latency metrics for the storage device; based on the time-distributed read/write latency performance profile and a characteristic time-distributed read/write latency performance profile for a representative group of storage devices having common characteristics with the storage device, determining that a probability that the storage device will enter a fault state exceeds a predetermined threshold; and based on the probability that the storage device exceeding the predetermined threshold, redirecting read/write requests for the storage device to other storage devices. ### Claim 9 (depends on 8) The computing device of claim 8, wherein the probability is determined in part on a difference between the time-distributed read/write latency performance profile for the storage device and the characteristic time-distributed read/write latency performance profile for the representative group of storage devices. ### Claim 10 (depends on 8) The computing device of claim 8, wherein the storage device is part of a group of storage devices that are configured to implement a fault resiliency function. ### Claim 11 (depends on 10) The computing device of claim 10, wherein the fault resiliency function is mirroring or erasure coding. ### Claim 12 (depends on 8) The computing device of claim 8, wherein the probability is determined in part on a time-based distribution of latencies for I/O operations associated with the storage device. ### Claim 13 (depends on 8) The computing device of claim 8, wherein the probability is determined in part on latencies for I/O operations associated with the storage device that have exceeded a threshold value. ### Claim 14 (depends on 8) The computing device of claim 8, further comprising determining that the probability that the storage device will enter the fault state has not exceeded the predetermined threshold for a specified time period. ### Claim 15 A non-transitory computer-readable medium having encoded thereon computer-executable instructions that, when executed, cause one or more processing units of a computing device to execute a method comprising: collecting read/write latency data for input/output operations executed at a storage device of a group of storage devices of a storage network; based on the collected read/write latency data, determining a time-distributed read/write latency performance profile for the storage device; determining a characteristic time-distributed read/write latency performance profile for a representative group of storage devices having common characteristics with the storage device; determining that a difference between the time-distributed read/write latency performance profile for the storage device and the characteristic time-distributed read/write latency performance profile exceeds a predetermined deviance threshold that is indicative of the storage device entering a fault state; and in response to determining that the difference exceeds the predetermined deviance threshold, redirecting read/write requests for the storage device to other storage devices. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the time-distributed read/write latency performance profile comprises a distribution of numbers of metrics that fell within each of a plurality of metric ranges. ### Claim 17 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the time-distributed read/write latency performance profile comprises a total number of I/O operations where associated latencies were greater than a threshold. ### Claim 18 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the time-distributed read/write latency performance profile for the storage device comprises a latency profile that is determined based on a number of latencies for the storage device that fall within each of a plurality of latency ranges and within a selected time period. ### Claim 19 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the group of storage devices are configured to implement a fault resiliency function.