# US 11,029,849 B2 · Handling cache and non-volatile storage (NVS) out of sync writes > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,029,849 B2 - Document ID: US11029849B2 - Status: Granted - Kind: B2 - Application number: 16/690,724 - Filed: 2019-11-21 - Published: 2020-03-19 - Granted: 2021-06-08 - Earliest claimed: 2017-09-05 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Kyler A. Anderson; Kevin J. Ash; Lokesh M. Gupta; Beth A. Peterson - Web page: https://patentgrep.com/patent/US11029849B2 ## CPC - G06F 3/0607: {by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device} - G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors} - G06F 3/065: {Replication mechanisms} - G06F 3/0656: {Data buffering arrangements} - G06F 3/0688: {Non-volatile semiconductor memory arrays} - G06F 12/0238: {Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory} - G06F 12/0868: Data transfer between cache memory and other subsystems, e.g. storage devices or host systems - G06F 2212/1032: Reliability improvement, data loss prevention, degraded operation etc - G06F 2212/154: Networked environment - G06F 2212/225: Hybrid cache memory, e.g. having both volatile and non-volatile portions - G06F 2212/2515: being configurable for different purposes, e.g. as cache or non-cache memory - G06F 2212/313: In storage device - G06F 2212/462: Track or segment - G06F 2212/7203: Temporary buffering, e.g. using volatile buffer or dedicated buffer blocks - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Provided are techniques for handling cache and Non-Volatile Storage (NVS) out of sync writes. At an end of a write for a cache track of a cache node, a cache node uses cache write statistics for the cache track of the cache node and Non-Volatile Storage (NVS) write statistics for a corresponding NVS track of an NVS node to determine that writes to the cache track and to the corresponding NVS track are out of sync. The cache node sets an out of sync indicator in a cache data control block for the cache track. The cache node sends a message to the NVS node to set an out of sync indicator in an NVS data control block for the corresponding NVS track. The cache node sets the cache track as pinned non-retryable due to the write being out of sync and reports possible data loss to error logs. ## Claims ### Claim 1 A computer program product, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by at least one processor to perform: at an end of a write for a Non-Volatile Storage (NVS) track of an NVS node, sending, with the NVS node, NVS write statistics to a cache node, wherein, at an end of a write for a corresponding cache track of the cache node, the cache node determines that writes to the corresponding cache track and to the NVS track are out of sync based on identifying a different number of the writes to the NVS track than to the corresponding cache track using the NVS write statistics and cache write statistics; receiving, with the NVS node, a message from the cache node to set an NVS out of sync indicator in an NVS data control block for the NVS track to indicate that the writes to the NVS track and the corresponding cache track are out of sync; setting, with the NVS node, the NVS out of sync indicator in the NVS data control block for the NVS track, wherein the cache node, sets a cache out of sync indicator in a cache data control block for the corresponding cache track to indicate that the writes to the NVS track and the corresponding cache track are out of sync; and discarding, with the NVS node, the NVS track, wherein the cache node sets the corresponding cache track as pinned non-retryable due to the write being out of sync. ### Claim 2 (depends on 1) The computer program product of claim 1, wherein the program code is executable by the at least one processor to perform: reporting, with the NVS node, possible data loss to error logs of the NVS node. ### Claim 3 (depends on 1) The computer program product of claim 1, wherein the cache node reports possible data loss to error logs of the cache node. ### Claim 4 (depends on 1) The computer program product of claim 1, wherein the cache node determines that any of a failover from the NVS node to the cache node has occurred and a warmstart has occurred prior to the cache node completing out of sync processing for the corresponding cache track, and, wherein the cache node informs the NVS node to discard the NVS track. ### Claim 5 (depends on 1) The computer program product of claim 1, wherein the program code is executable by the at least one processor to perform: determining, with the NVS node, that failover from the cache node to the NVS node has occurred prior to the NVS node completing out of sync processing; and sending, with the NVS node, the NVS data control block for the NVS track to the cache node to enable the cache node to create a new cache data control block, wherein the cache node completes the out of sync processing. ### Claim 6 (depends on 1) The computer program product of claim 1, wherein the program code is executable by the at least one processor to perform: during an initial microcode load, with the NVS node, examining each NVS data control block storing an NVS out of sync indicator, which was preserved across a reboot, to determine whether the NVS out of sync indicator is set in that NVS data control block; and for each NVS data control block with the NVS out of sync indicator set, using that NVS data control block to complete out of sync processing. ### Claim 7 A computer system, comprising: one or more processors, one or more computer-readable memories and one or more computer-readable, tangible storage devices; and program instructions, stored on at least one of the one or more computer-readable, tangible storage devices for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, to perform operations comprising: at an end of a write for a Non-Volatile Storage (NVS) track of an NVS node, sending, with the NVS node, NVS write statistics to a cache node, wherein, at an end of a write for a corresponding cache track of the cache node, the cache node determines that writes to the corresponding cache track and to the NVS track are out of sync based on identifying a different number of the writes to the NVS track than to the corresponding cache track using the NVS write statistics and cache write statistics; receiving, with the NVS node, a message from the cache node to set an NVS out of sync indicator in an NVS data control block for the NVS track to indicate that the writes to the NVS track and the corresponding cache track are out of sync; setting, with the NVS node, the NVS out of sync indicator in the NVS data control block for the NVS track, wherein the cache node, sets a cache out of sync indicator in a cache data control block for the corresponding cache track to indicate that the writes to the NVS track and the corresponding cache track are out of sync; and discarding, with the NVS node, the NVS track, wherein the cache node sets the corresponding cache track as pinned non-retryable due to the write being out of sync. ### Claim 8 (depends on 7) The computer system of claim 7, wherein the operations further comprise: reporting, with the NVS node, possible data loss to error logs of the NVS node. ### Claim 9 (depends on 7) The computer system of claim 7, wherein the cache node reports possible data loss to error logs of the cache node. ### Claim 10 (depends on 7) The computer system of claim 7, wherein the cache node determines that any of a failover from the NVS node to the cache node has occurred and a warmstart has occurred prior to the cache node completing out of sync processing for the corresponding cache track, and, wherein the cache node informs the NVS node to discard the NVS track. ### Claim 11 (depends on 7) The computer system of claim 7, wherein the operations further comprise: determining, with the NVS node, that failover from the cache node to the NVS node has occurred prior to the NVS node completing out of sync processing; and sending, with the NVS node, the NVS data control block for the NVS track to the cache node to enable the cache node to create a new cache data control block, wherein the cache node completes the out of sync processing. ### Claim 12 (depends on 7) The computer system of claim 7, wherein the operations further comprise: during an initial microcode load, with the NVS node, examining each NVS data control block storing an NVS out of sync indicator, which was preserved across a reboot, to determine whether the NVS out of sync indicator is set in that NVS data control block; and for each NVS data control block with the NVS out of sync indicator set, using that NVS data control block to complete out of sync processing. ### Claim 13 A computer-implemented method, comprising: at an end of a write for a Non-Volatile Storage (NVS) track of an NVS node, sending, with the NVS node, NVS write statistics to a cache node, wherein, at an end of a write for a corresponding cache track of the cache node, the cache node determines that writes to the corresponding cache track and to the NVS track are out of sync based on identifying a different number of the writes to the NVS track than to the corresponding cache track using the NVS write statistics and cache write statistics; receiving, with the NVS node, a message from the cache node to set an NVS out of sync indicator in an NVS data control block for the NVS track to indicate that the writes to the NVS track and the corresponding cache track are out of sync; setting, with the NVS node, the NVS out of sync indicator in the NVS data control block for the NVS track, wherein the cache node, sets a cache out of sync indicator in a cache data control block for the corresponding cache track to indicate that the writes to the NVS track and the corresponding cache track are out of sync; and discarding, with the NVS node, the NVS track, wherein the cache node sets the corresponding cache track as pinned non-retryable due to the write being out of sync. ### Claim 14 (depends on 13) The computer-implemented method of claim 13, further comprising: reporting, with the NVS node, possible data loss to error logs of the NVS node. ### Claim 15 (depends on 13) The computer-implemented method of claim 13, wherein the cache node reports possible data loss to error logs of the cache node. ### Claim 16 (depends on 13) The computer-implemented method of claim 13, wherein the cache node determines that any of a failover from the NVS node to the cache node has occurred and a warmstart has occurred prior to the cache node completing out of sync processing for the corresponding cache track, and, wherein the cache node informs the NVS node to discard the NVS track. ### Claim 17 (depends on 13) The computer-implemented method of claim 13, further comprising: determining, with the NVS node, that failover from the cache node to the NVS node has occurred prior to the NVS node completing out of sync processing; and sending, with the NVS node, the NVS data control block for the NVS track to the cache node to enable the cache node to create a new cache data control block, wherein the cache node completes the out of sync processing. ### Claim 18 (depends on 13) The computer-implemented method of claim 13, further comprising: during an initial microcode load, with the NVS node, examining each NVS data control block storing an NVS out of sync indicator, which was preserved across a reboot, to determine whether the NVS out of sync indicator is set in that NVS data control block; and for each NVS data control block with the NVS out of sync indicator set, using that NVS data control block to complete out of sync processing.