# US 9,519,540 B2 · Apparatus, system, and method for destaging cached data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,540 B2 - Document ID: US9519540B2 - Status: Granted - Kind: B2 - Application number: 13/357,465 - Filed: 2012-01-24 - Published: 2012-05-17 - Granted: 2016-12-13 - Earliest claimed: 2007-12-06 - Assignees: SanDisk Technologies LLC - Applicants: David Atkisson; Jonathan Ludwig - Inventors: David Atkisson; Jonathan Ludwig - Web page: https://patentgrep.com/patent/US9519540B2 ## CPC - G06F 11/108: {Parity data distribution in semiconductor storages, e.g. in SSD} - G06F 12/0804: with main memory updating (G06F12/0806 takes precedence) - G06F 12/0868: Data transfer between cache memory and other subsystems, e.g. storage devices or host systems - G06F 13/28: using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence) - G06F 13/4022: {using switching circuits, e.g. switching matrix, connection or expansion network (G06F13/4009 takes precedence)} - H04L 12/403: with centralised control, e.g. polling - H04L 12/4625: {Single bridge functionality, e.g. connection of two networks over a single bridge} - H04L 12/4633: {Interconnection of networks using encapsulation techniques, e.g. tunneling} - H05K 7/1444: {Complex or three-dimensional-arrangements; Stepped or dual mother boards} - G06F 3/0656: {Data buffering arrangements} - G06F 3/0688: {Non-volatile semiconductor memory arrays} - G06F 2211/103: Hybrid, i.e. RAID systems with parity comprising a mix of RAID types - G06F 2212/222: Non-volatile memory - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H05K: PRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS ## Abstract An apparatus, system, and method are disclosed for satisfying storage requests while destaging cached data. A monitor module samples a destage rate for a nonvolatile solid-state cache, a total cache write rate for the cache, and a dirtied data rate. The dirtied data rate comprises a rate at which write operations increase an amount of dirty data in the cache. A target module determines a target cache write rate for the cache based on the destage rate, the total cache write rate, and the dirtied data rate to target a destage write ratio. The destage write ratio comprises a predetermined ratio between the dirtied data rate and the destage rate. A rate enforcement module enforces the target cache write rate such that the total cache write rate satisfies the target cache write rate. ## Claims ### Claim 1 A method for satisfying storage requests while destaging cached data, the method comprising: sampling a destage rate for a nonvolatile solid-state cache, a total cache write rate for the cache, and a dirtied data rate, the destage rate comprising a rate at which dirty data is destaged from the cache, the total cache write rate comprising a total amount of data written to the cache in response to user write requests, the total cache write rate further comprising a dirty write hit rate and a clean write hit rate, the dirty write hit rate comprising a rate at which cache writes invalidate dirty data in the cache, the clean write hit rate comprising a rate at which cache writes invalidate previously destaged clean data in the cache, the dirtied data rate comprising a rate at which write operations increase an amount of dirty data in the cache; determining a target cache write rate for the cache based on the destage rate, the total cache write rate, and the dirtied data rate to target a destage write ratio, the destage write ratio comprising a predetermined ratio between the dirtied data rate and the destage rate; and enforcing the target cache write rate such that the total cache write rate satisfies the target cache write rate. ### Claim 2 (depends on 1) The method of claim 1, further comprising, sampling a total user read rate and a total backing store read rate for the cache; determining a target user read rate based on the destage rate, the total user read rate, and the total backing store read rate to target a destage read ratio, the destage read ratio comprising a predetermined ratio between the total backing store read rate and the destage rate; and enforcing the target user read rate such that the total user read rate satisfies the target user read rate. ### Claim 3 (depends on 2) The method of claim 2, wherein the target user read rate comprises a product of the destage read ratio; the destage rate divided by the backing store read rate; and the total user read rate. ### Claim 4 (depends on 2) The method of claim 2, wherein a difference in magnitude of the destage read ratio relative to the destage write ratio is selected based on relative priorities of read operations and write operations during destaging of the cache. ### Claim 5 (depends on 1) The method of claim 1, wherein the target cache write rate exceeds the destage rate for the cache in response to at least a portion of the total cache write rate for the cache comprising updates to dirty data in the cache that do not increase the amount of dirty data in the cache. ### Claim 6 (depends on 1) The method of claim 1, further comprising triggering a write rate enforcement operation in response to a data flush operation for the cache, the write rate enforcement operation comprising the sampling, determining, and enforcing steps. ### Claim 7 (depends on 6) The method of claim 6, further comprising servicing write requests at or below the target cache write rate during the data flush operation. ### Claim 8 (depends on 6) The method of claim 6, further comprising adjusting the destage write ratio over time such that the target cache write rate satisfies a minimum write rate threshold as the dirtied data rate increases during the data flush operation. ### Claim 9 (depends on 6) The method of claim 6, further comprising selecting the destage write ratio such that the data flush operation completes within a predetermined flush operation window. ### Claim 10 (depends on 6) The method of claim 6, further comprising, quiescing write operations for the cache to complete the data flush operation in response to the amount of dirty data in the cache falling below a completion threshold; transitioning the cache from a write-back mode to one of a write-through mode and a write-around mode in response to completing the data flush operation; and resuming servicing of write operations using a backing store of the cache in response to transitioning the cache from the write-back mode. ### Claim 11 (depends on 10) The method of claim 10, further comprising, performing a maintenance operation on the backing store of the cache in response to completing the data flush operation; and transitioning the cache back to the write-back mode in response to completing the maintenance operation. ### Claim 12 (depends on 1) The method of claim 1, further comprising triggering a write rate enforcement operation during normal runtime of the cache in response to the amount of dirty data in the cache failing to satisfy a dirty data threshold, the write rate enforcement operation comprising the sampling, determining, and enforcing steps. ### Claim 13 (depends on 1) The method of claim 1, wherein the target cache write rate comprises a product of the destage write ratio; the destage rate divided by the dirtied data rate; and the total cache write rate. ### Claim 14 (depends on 1) The method of claim 1, wherein enforcing the target cache write rate comprises quiescing write operations for the cache in response to the total cache write rate exceeding the target cache write rate. ### Claim 15 (depends on 1) The method of claim 1, further comprising repeating sampling the destage rate, the total cache write rate, and the dirtied data rate and determining the target cache write rate for each of a plurality of sampling periods, wherein enforcing the target cache write rate comprises enforcing, during one sampling period, the target cache write rate determined for a previous sampling period. ### Claim 16 (depends on 1) The method of claim 1, wherein the destage write ratio is defined by a user and represents a predetermined priority between servicing write requests and destaging data from the cache. ### Claim 17 An apparatus for satisfying storage requests while destaging cached data, the apparatus comprising: a monitor module that samples a destage rate for a nonvolatile solid-state cache, a total cache write rate for the cache, and a dirtied data rate, the destage rate comprising a rate at which dirty data is destaged from the cache, the total cache write rate comprising a total amount of data written to the cache in response to user write requests, the total cache write rate further comprising a dirty write hit rate and a clean write hit rate, the dirty write hit rate comprising a rate at which cache writes invalidate dirty data in the cache, the clean write hit rate comprising a rate at which cache writes invalidate previously destaged clean data in the cache, the dirtied data rate comprising a rate at which write operations increase an amount of dirty data in the cache; a target module that determines a target cache write rate for the cache based on the destage rate, the total cache write rate, and the dirtied data rate to target a destage write ratio, the destage write ratio comprising a predetermined ratio between the dirtied data rate and the destage rate; and a rate enforcement module that enforces the target cache write rate such that the total cache write rate satisfies the target cache write rate. ### Claim 18 (depends on 17) The apparatus of claim 17, wherein, the monitor module samples a total user read rate and a total backing store read rate for the cache; the target module determines a target user read rate based on the destage rate, the total user read rate, and the total backing store read rate to target a destage read ratio, the destage read ratio comprising a predetermined ratio between the total backing store read rate and the destage rate; and the rate enforcement module enforces the target user read rate such that the total user read rate satisfies the target user read rate. ### Claim 19 A system for satisfying storage requests while destaging cached data, the system comprising: a nonvolatile solid-state cache; a backing store for the nonvolatile solid-state cache; a cache controller for the nonvolatile solid-state cache, the cache controller comprising, a monitor module that samples a destage rate from the cache to the backing store, a total cache write rate for the cache, and a dirtied data rate, the destage rate comprising a rate at which dirty data is destaged from the cache, the total cache write rate comprising a total amount of data written to the cache in response to user write requests, the total cache write rate further comprising a dirty write hit rate and a clean write hit rate, the dirty write hit rate comprising a rate at which cache writes invalidate dirty data in the cache, the clean write hit rate comprising a rate at which cache writes invalidate previously destaged clean data in the cache, the dirtied data rate comprising a rate at which write operations increase an amount of dirty data in the cache; a target module that determines a target cache write rate for the cache based on the destage rate, the total cache write rate, and the dirtied data rate to target a destage write ratio, the destage write ratio comprising a predetermined ratio between the dirtied data rate and the destage rate; and a rate enforcement module that enforces the target cache write rate such that the total cache write rate satisfies the target cache write rate. ### Claim 20 (depends on 19) The system of claim 19, further comprising a host computer system in communication with the cache and the backing store, the cache controller comprising a device driver executing on the host computer system.