# US 11,442,825 B2 · Establishing a synchronous replication relationship between two or more storage systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,442,825 B2 - Document ID: US11442825B2 - Status: Granted - Kind: B2 - Application number: 17/240,334 - Filed: 2021-04-26 - Published: 2021-10-21 - Granted: 2022-09-13 - Earliest claimed: 2017-03-10 - Assignees: Pure Storage, Inc. - Applicants: Pure Storage, Inc. - Inventors: David Grunwald; Steven Hodgson; Ronald Karr; Kunal Trivedi; Christopher Golden; Thomas Gill; Connor Brooks; Zoheb Shivani - Web page: https://patentgrep.com/patent/US11442825B2 ## CPC - G06F 3/0616: {in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]} - G06F 11/2076: {Synchronous techniques} - G06F 3/0604: {Improving or facilitating administration, e.g. storage management} - G06F 3/061: {Improving I/O performance} - G06F 3/0632: {by initialisation or re-initialisation of storage systems} - G06F 3/065: {Replication mechanisms} - G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling} - G06F 3/0683: {Plurality of storage devices} - G06F 3/0688: {Non-volatile semiconductor memory arrays} - G06F 9/44505: {Configuring for program initiating, e.g. using registry, configuration files} - G06F 11/0727: {in a storage system, e.g. in a DASD or network based storage system (drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097)} - G06F 11/0751: {Error or fault detection not based on redundancy (power supply failures G06F1/30; network fault management H04L41/06)} - G06F 11/1464: {for networked environments} - G06F 11/2064: {while ensuring consistency} - G06F 11/2082: {Data synchronisation} - G06F 11/2094: {Redundant storage or storage space (G06F11/2056 takes precedence)} - G06F 11/2097: {maintaining the standby controller/processing unit updated (initialisation or re-synchronisation thereof G06F11/1658 and subgroups)} - G06F 12/0684: {with feedback, e.g. presence or absence of unit detected by addressing, overflow detection} - G06F 12/1072: Decentralised address translation, e.g. in distributed shared memory systems - G06F 16/178: Techniques for file synchronisation in file systems - G06F 16/182: Distributed file systems - G06F 16/1844: {Management specifically adapted to replicated file systems} - G06F 16/27: Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor - G06F 16/275: {Synchronous replication} - H04L 45/12: Shortest path evaluation - G06F 3/06: Digital input from, or digital output to, record carriers {, e.g. RAID, emulated record carriers or networked record carriers} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 11/2053: {where persistent mass storage functionality or persistent mass storage control functionality is redundant (error detection or correction in information storage based on relative movement between record carrier and transducer G11B20/18)} - H04L 45/38: {Flow based routing} - H04L 47/125: by balancing the load, e.g. traffic engineering - H04L 67/1095: Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Establishing a synchronous replication relationship between two or more storage systems, including: identifying, for a dataset, a plurality of storage systems across which the dataset will be synchronously replicated; configuring one or more data communications links between each of the plurality of storage systems to be used for synchronously replicating the dataset; exchanging, between the plurality of storage systems, timing information for at least one of the plurality of storage systems; and establishing, in dependence upon the timing information for at least one of the plurality of storage systems, a synchronous replication lease, the synchronous replication lease identifying a period of time during which the synchronous replication relationship is valid. ## Claims ### Claim 1 A method comprising: exchanging, between a plurality of storage systems across which a dataset will be synchronously replicated, timing information for at least one of the plurality of storage systems; and establishing, in dependence upon the timing information for at least one of the plurality of storage systems, a synchronous replication lease, the synchronous replication lease identifying a period of time during which the synchronous replication relationship is valid, wherein a request to modify the dataset may only be acknowledged after a copy of the dataset has been modified on each of the storage systems. ### Claim 2 (depends on 1) The method of claim 1 wherein the synchronous replication lease is calculated in dependence upon a value of a local clock that was received from another storage system. ### Claim 3 (depends on 1) The method of claim 1 further comprising wherein establishing the synchronous replication lease further comprises further comprises exchanging uncoordinated clocks between a plurality of storage systems. ### Claim 4 (depends on 1) The method of claim 1 further comprising re-establishing a synchronous replication relationship after detecting that the synchronous replication lease has expired. ### Claim 5 (depends on 1) The method of claim 1 further comprising attempting to take over I/O processing for the dataset after detecting that the synchronous replication lease has expired. ### Claim 6 (depends on 1) The method of claim 1 wherein at least one of the plurality of storage systems is a cloud-based storage system. ### Claim 7 (depends on 1) The method of claim 1 wherein the timing information for at least one of the plurality of storage systems includes a value which serves as a proxy for a clock value. ### Claim 8 (depends on 1) The method of claim 1 further comprising delaying the processing of I/O requests received after the synchronous replication lease has expired. ### Claim 9 (depends on 1) The method of claim 1 further comprising establishing a cluster membership lease. ### Claim 10 An apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of: exchanging, between a plurality of storage systems across which a dataset will be synchronously replicated, timing information for at least one of the plurality of storage systems; and establishing, in dependence upon the timing information for at least one of the plurality of storage systems, a synchronous replication lease, the synchronous replication lease identifying a period of time during which the synchronous replication relationship is valid, wherein a request to modify the dataset may only be acknowledged after a copy of the dataset has been modified on each of the storage systems. ### Claim 11 (depends on 10) The apparatus of claim 10 wherein the synchronous replication lease is calculated in dependence upon a value of a local clock that was received from another storage system. ### Claim 12 (depends on 10) The apparatus of claim 10 wherein establishing the synchronous replication lease further comprises further comprises exchanging uncoordinated clocks between a plurality of storage systems. ### Claim 13 (depends on 10) The apparatus of claim 10 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out re-establishing a synchronous replication relationship after detecting that the synchronous replication lease has expired. ### Claim 14 (depends on 10) The apparatus of claim 10 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out attempting to take over I/O processing for the dataset after detecting that the synchronous replication lease has expired. ### Claim 15 (depends on 10) The apparatus of claim 10 wherein at least one of the plurality of storage systems is a cloud-based storage system. ### Claim 16 A storage system comprising a computer processor and a computer memory operatively coupled to the computer processor, the computer memory having disposed within it computer program instructions that, when executed by the computer processor, cause the storage system to carry out the steps of: sending, to a second storage system, timing information for the storage system; receiving, from the second storage system, timing information for the second storage system; and establishing a synchronous replication lease, the synchronous replication lease identifying a period of time during which the synchronous replication relationship is valid, wherein a request to modify the dataset may only be acknowledged after a copy of the dataset has been modified on each of the storage systems. ### Claim 17 (depends on 16) The storage system of claim 16 further comprising computer program instructions that, when executed by the computer processor, cause the storage system to carry out the step of setting, in dependence upon the timing information for the second storage system, a clock value in the storage system. ### Claim 18 (depends on 16) The storage system of claim 16 wherein the storage system is a cloud-based storage system. ### Claim 19 (depends on 16) The storage system of claim 16 wherein sending, to the second storage system, timing information for the storage system includes sending, to the second storage system, a most recently received value of a clock in the second storage system. ### Claim 20 (depends on 16) The storage system of claim 16 further comprising computer program instructions that, when executed by the computer processor, cause the storage system to carry attempting to take over I/O processing for the dataset after detecting that the synchronous replication lease has expired.