# US 11,307,772 B1 · Responding to variable response time behavior in a storage environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,307,772 B1 - Document ID: US11307772B1 - Status: Granted - Kind: B1 - Application number: 15/221,686 - Filed: 2016-07-28 - Granted: 2022-04-19 - Earliest claimed: 2010-09-15 - Assignees: Pure Storage, Inc. - Applicants: Pure Storage, Inc. - Inventors: John Colgrove; John Hayes; Bo Hong; Feng Wang; Ethan Miller; Craig Harmer - Web page: https://patentgrep.com/patent/US11307772B1 ## CPC - G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors} - G06F 3/061: {Improving I/O performance} - G06F 3/0611: {in relation to response time} - G06F 3/0613: {in relation to throughput} - G06F 3/0629: {Configuration or reconfiguration of storage systems} - G06F 3/0646: {Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems} - G06F 3/0653: {Monitoring storage devices or systems} - G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling} - G06F 3/0665: {at area level, e.g. provisioning of virtual or logical volumes} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 3/0685: {Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays} - G06F 3/0688: {Non-volatile semiconductor memory arrays} - G06F 3/0689: {Disk arrays, e.g. RAID, JBOD} - G06F 11/0727: {in a storage system, e.g. in a DASD or network based storage system (drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097)} - G06F 11/0751: {Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06)} - G06F 11/1076: {Parity data used in redundant arrays of independent storages, e.g. in RAID systems} - G06F 12/0804: with main memory updating (G06F12/0806 takes precedence) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system and method for effectively scheduling read and write operations among a plurality of solid-state storage devices. A computer system comprises client computers and data storage arrays coupled to one another via a network. A data storage array utilizes solid-state drives and Flash memory cells for data storage. A storage controller within a data storage array comprises an I/O scheduler. The storage controller is configured to receive a read request targeted to the data storage medium, and identify at least a first storage device of the plurality of storage devices which contains data targeted by the read request. In response to either detecting or predicting the first storage device will exhibit variable performance, the controller is configured to generate a reconstruct read request configured to obtain the data from one or more devices of the plurality of storage devices other than the first storage device. ## Claims ### Claim 1 A method comprising: by a data storage controller: receiving a read request for data on a first device; prior to issuing the read request to the first device, determining that servicing of a reconstruct read request for the data by a second device can complete prior to servicing of the read request by the first device; prior to issuing the read request to the first device and in response to determining that servicing of the reconstruct read request for the data by the second device can complete prior to servicing of the read request by the first device, initiating the reconstruct read request configured to obtain the data from the second device. ### Claim 2 (depends on 1) The method of claim 1 wherein determining whether to issue the read request to the first device includes: determining whether the second device is predicted to continue exhibiting variable response time behavior after the first device is predicted to stop exhibiting variable response time behavior; in response to determining that the second device is predicted to continue exhibiting variable response time behavior after the first device is predicted to stop exhibiting variable response time behavior, servicing the read request by issuing the read request to the first device; and in response to determining that the second device is not predicted to continue exhibiting variable response time behavior after the first device is predicted to stop exhibiting variable response time behavior, servicing the read request with the reconstruct read request to the second device. ### Claim 3 (depends on 1) The method of claim 1 further comprising: in response to determining that the second device is not exhibiting variable response time behavior, determining whether the first device is not exhibiting variable response time behavior; in response to determining that the first device is exhibiting variable response time behavior, servicing the read request with the reconstruct read request to the second device. ### Claim 4 (depends on 3) The method of claim 3 further comprising: in response to determining that the first device is not exhibiting variable response time behavior: issuing the read request to the first device; determining whether the read request to the first device returns a first result before the reconstruct read request to the second device returns a second result; in response to determining that the read request to the first device returns the first result before the reconstruct read request to the second device returns the second result, servicing the read request with the first result; and in response to determining that the read request to the first device does not return the first result before the reconstruct read request to the second device returns the second result, servicing the read request with the second result. ### Claim 5 (depends on 4) The method of claim 4 further comprising in response to determining that the read request to the first device returns the first result before the reconstruct read request to the second device returns the second result, discarding the second result. ### Claim 6 (depends on 4) The method of claim 4 further comprising in response to determining that the read request to the first device returns the first result before the reconstruct read request to the second device returns the second result, removing the reconstructive read request from the second device. ### Claim 7 (depends on 4) The method of claim 4 further comprising in response to determining that the read request to the first device does not return the first result before the reconstruct read request to the second device returns the second result, discarding the first result. ### Claim 8 (depends on 4) The method of claim 4 further comprising in response to determining that the read request to the first device does not return the first result before the reconstruct read request to the second device returns the second result, removing the read request from the first device. ### Claim 9 An apparatus comprising a computer processor and a computer memory, the computer memory including computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of: by a data storage controller: receiving a read request for data on a first device; prior to issuing the read request to the first device, determining that servicing of a reconstruct read request for the data by a second device can complete prior to servicing of the read request by the first device; prior to issuing the read request to the first device and in response to determining that servicing of the reconstruct read request for the data by the second device can complete prior to servicing of the read request by the first device, initiating the reconstruct read request configured to obtain the data from the second device. ### Claim 10 (depends on 9) The apparatus of claim 9 wherein determining whether to issue the read request to the first device includes: determining whether the second device is predicted to continue exhibiting variable response time behavior after the first device is predicted to stop exhibiting variable response time behavior; in response to determining that the second device is predicted to continue exhibiting variable response time behavior after the first device is predicted to stop exhibiting variable response time behavior, servicing the read request by issuing the read request to the first device; and in response to determining that the second device is not predicted to continue exhibiting variable response time behavior after the first device is predicted to stop exhibiting variable response time behavior, servicing the read request with the reconstruct read request to the second device. ### Claim 11 (depends on 9) The apparatus of claim 9 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of: in response to determining that the second device is not exhibiting variable response time behavior, determining whether the first device is not exhibiting variable response time behavior; in response to determining that the first device is exhibiting variable response time behavior, servicing the read request with the reconstruct read request to the second device. ### Claim 12 (depends on 11) The apparatus of claim 11 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of: in response to determining that the first device is not exhibiting variable response time behavior: issuing the read request to the first device; determining whether the read request to the first device returns a first result before the reconstruct read request to the second device returns a second result; in response to determining that the read request to the first device returns the first result before the reconstruct read request to the second device returns the second result, servicing the read request with the first result; and in response to determining that the read request to the first device does not return the first result before the reconstruct read request to the second device returns the second result, servicing the read request with the second result. ### Claim 13 (depends on 12) The apparatus of claim 12 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of in response to determining that the read request to the first device returns the first result before the reconstruct read request to the second device returns the second result, discarding the second result. ### Claim 14 (depends on 13) The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of in response to determining that the read request to the first device returns the first result before the reconstruct read request to the second device returns the second result, removing the reconstructive read request from the second device. ### Claim 15 (depends on 13) The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of in response to determining that the read request to the first device does not return the first result before the reconstruct read request to the second device returns the second result, discarding the first result. ### Claim 16 (depends on 13) The apparatus of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the step of in response to determining that the read request to the first device does not return the first result before the reconstruct read request to the second device returns the second result, removing the read request from the first device. ### Claim 17 A computer program product including a non-transitory computer readable medium, the computer program product comprising computer program instructions that, when executed, cause a computer to carry out the steps of: by a data storage controller: receiving a read request for data on a first device; prior to issuing the read request to the first device, determining that servicing of a reconstruct read request for the data by a second device can complete prior to servicing of the read request by the first device; prior to issuing the read request to the first device and in response to determining that servicing of the reconstruct read request for the data by the second device can complete prior to servicing of the read request by the first device, initiating the reconstruct read request configured to obtain the data from the second device. ### Claim 18 (depends on 17) The computer program product of claim 17 wherein determining whether to issue the read request to the first device includes: determining whether the second device is predicted to continue exhibiting variable response time behavior after the first device is predicted to stop exhibiting variable response time behavior; in response to determining that the second device is predicted to continue exhibiting variable response time behavior after the first device is predicted to stop exhibiting variable response time behavior, servicing the read request by issuing the read request to the first device; and in response to determining that the second device is not predicted to continue exhibiting variable response time behavior after the first device is predicted to stop exhibiting variable response time behavior, servicing the read request with the reconstruct read request to the second device. ### Claim 19 (depends on 17) The computer program product of claim 17 further comprising computer program instructions that, when executed, cause a computer to carry out the steps of: in response to determining that the second device is not exhibiting variable response time behavior, determining whether the first device is not exhibiting variable response time behavior; in response to determining that the first device is exhibiting variable response time behavior, servicing the read request with the reconstruct read request to the second device. ### Claim 20 (depends on 19) The computer program product of claim 19 further comprising computer program instructions that, when executed, cause a computer to carry out the steps of: in response to determining that the first device is not exhibiting variable response time behavior: issuing the read request to the first device; determining whether the read request to the first device returns a first result before the reconstruct read request to the second device returns a second result; in response to determining that the read request to the first device returns the first result before the reconstruct read request to the second device returns the second result, servicing the read request with the first result; and in response to determining that the read request to the first device does not return the first result before the reconstruct read request to the second device returns the second result, servicing the read request with the second result.