# US 9,519,657 B2 · Clustered filesystem with membership version support > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,657 B2 - Document ID: US9519657B2 - Status: Granted - Kind: B2 - Application number: 14/199,619 - Filed: 2014-03-06 - Published: 2014-07-03 - Granted: 2016-12-13 - Earliest claimed: 2001-06-05 - Assignees: Silicon Graphics International Corp. - Applicants: Silicon Graphics International Corp. - Inventors: Kenneth S. Beck - Web page: https://patentgrep.com/patent/US9519657B2 ## CPC - G06F 16/182: Distributed file systems - G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors} - G06F 3/065: {Replication mechanisms} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 11/2064: {while ensuring consistency} - G06F 11/2082: {Data synchronisation} - G06F 16/10: File systems; File servers - G06F 16/128: {Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion (error detection or correction of the data by redundancy in operations or in hardware G06F11/14, G06F11/16)} - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - H04L 67/564: Enhancement of application control based on intercepted application data - H04L 67/566: Grouping or aggregating service requests, e.g. for unified processing - G06F 2201/84: Using snapshots, i.e. a logical point-in-time copy of the data - Y10S 707/966: Distributed - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract A computer system with read/write access to storage devices creates a snapshot of a data volume at a point in time while continuing to accept access requests to the mirrored data volume by copying before making changes to the base data volume. Multiple snapshots may be made of the same data volume at different points in time. Only data that is not stored in a previous snapshot volume or in the base data volume are stored in the most recent snapshot volume. ## Claims ### Claim 1 A method for establishing cluster membership of computer system nodes sharing read/write access to storage devices via a storage area network, the method comprising: sending a withdrawal request from a node in a viable quorum cluster to two or more other nodes in the viable quorum cluster when the node is scheduled to withdraw from the viable quorum cluster to upgrade a version of an operating system software at the node, wherein the withdrawal request is sent according to a rolling upgrade, the rolling upgrade performed such that at least one other node of the two or more other nodes in the viable quorum cluster can receive the upgraded version of the operating system software at a later point in time, and the node withdraws from membership in the viable quorum cluster after sending the withdrawal request; upgrading the version of the operating system software program at the node; receiving a request for information from a leader node; providing information identifying the upgraded version of the operating system software at the node to the leader node, wherein the leader node aggregates the upgraded operating system software version information and provides the aggregated information to all nodes in the viable quorum cluster when proposing a renewed membership in the viable quorum cluster and re-joining the viable quorum cluster after the leader node provides the aggregated information to all the nodes in the viable quorum cluster after the renewed membership in the viable quorum cluster has been proposed, wherein the renewed membership in the viable quorum cluster includes the node that previously withdrew from the viable quorum cluster and at least one other node, and the at least one other node formats a message consistent with the upgraded version of the operating system software at the node for transmission to the node. ### Claim 2 (depends on 1) The method of claim 1, further comprising receiving an acknowledgement at the node before the node withdraws from the viable quorum cluster, wherein the acknowledgement indicates that a new membership of the viable quorum cluster has been established including at least two nodes of the two or more other nodes. ### Claim 3 (depends on 2) The method of claim 2, wherein the node transitions to a shutdown state when withdrawing from the viable quorum cluster. ### Claim 4 (depends on 1) The method of claim 1, further comprising: entering a nascent state by the node; and sending heartbeat signals from the node in the nascent state, wherein the node re-joins the viable quorum cluster after sending the heartbeat signals. ### Claim 5 (depends on 4) The method of claim 4, further comprising transmitting version tags from the node, wherein the version tags correspond to a version level of a function supported by the node. ### Claim 6 (depends on 5) The method of claim 5, wherein the version tags correspond to a message format compatible with the version level of the function. ### Claim 7 (depends on 5) The method of claim 5, wherein the function supported is a distributed algorithm. ### Claim 8 (depends on 5) The method of claim 5, wherein the function supported is the software program. ### Claim 9 A system for establishing cluster membership of computer system nodes sharing read/write access to storage devices via a storage area network, the system comprising: a plurality of compute nodes, wherein the plurality of compute nodes form a viable quorum cluster; and a data communication network, wherein a node sends a withdrawal request to two or more other nodes in the viable quorum cluster when the node intends to withdraw from the viable quorum cluster to upgrade a version of an operating system software at the node, the withdrawal request sent through the data communication network, wherein: the withdrawal request is sent according to a rolling upgrade, the rolling upgrade performed such that at least one other node of the plurality of compute nodes in the viable quorum cluster can receive the upgraded version of the operating system software at a later point in time, the node withdraws from membership in the viable quorum cluster after sending the withdrawal request, the version of the operating system software program at the node is upgraded, a request for information is received from a leader node; information identifying the upgraded version of the operating system at the node is provided to the leader node, the leader node aggregates the upgraded operating system software version information and provides the aggregated information to all nodes in the viable quorum cluster when proposing a renewed membership in the viable quorum, the node re-joins the viable quorum cluster after the leader node aggregates the upgraded operating system software version information and provides the aggregated information to all nodes in the viable quorum cluster when proposing a renewed membership in the viable quorum, and the renewed membership in the viable quorum cluster includes the node that previously withdrew from the viable quorum cluster and at least one other node, and the at least one other node formats a message consistent with the upgraded version of the operating system software at the node for transmission to the node. ### Claim 10 (depends on 9) The system of claim 9, wherein the node receives an acknowledgement before the node withdraws from the viable quorum cluster through the data communication network, and the acknowledgement indicates that a new membership of the viable quorum cluster has been established including at least two nodes of the two or more other nodes. ### Claim 11 (depends on 10) The system of claim 10, wherein the node transitions to a shutdown state when withdrawing from the viable quorum cluster. ### Claim 12 (depends on 9) The system of claim 9, wherein the node: enters a nascent state; and sends heartbeat signals from when in the nascent state, and re-joins the viable quorum cluster after sending the heartbeat signals. ### Claim 13 (depends on 12) The system of claim 12, wherein the node transmits version tags, and the version tags correspond to a version level of a function supported by the node. ### Claim 14 (depends on 13) The system of claim 13, wherein the version tags correspond to a message format compatible with the version level of the function. ### Claim 15 (depends on 13) The system of claim 13, wherein the function supported is a distributed algorithm. ### Claim 16 (depends on 13) The system of claim 13, wherein the function supported is the software program. ### Claim 17 A non-transitory computer readable storage medium having embodied thereon a program for performing a method for establishing cluster membership of computer system nodes sharing read/write access to storage devices via a storage area network, the method comprising: sending a withdrawal request from a node in a viable quorum cluster to two or more other nodes in the viable quorum cluster when the node is scheduled to withdraw from the viable quorum cluster to upgrade a version of an operating system software at the node, wherein the withdrawal request is sent according to a rolling upgrade, the rolling upgrade performed such that at least one other node of the two or more other nodes in the viable quorum cluster can receive the upgraded version of the operating system software at a later point in time, and the node withdraws from membership in the viable quorum cluster after sending the withdrawal request; upgrading the version of an the version of the operating system software program at the node; receiving a request for information from a leader node; providing information identifying the upgraded version of the operating system software at the node to the leader node; and re-joining the viable quorum cluster after the leader node provides the aggregated information to all the nodes in the viable quorum cluster after the renewed membership in the viable quorum cluster has been proposed, wherein the rewnewed membership in the vialble quorum cluster includes the node that previously withdrew from the viable quorum cluster and at least one other node, and the at least one other node formats a message consistent with the upgraded version of the operating system software at the node for transmission to the node. ### Claim 18 (depends on 17) The non-transitory computer readable storage medium of claim 17, the program further executable to: enter a nascent state by the node; and send heartbeat signals from the node in the nascent state, wherein the node re-joins the viable quorum cluster after sending the heartbeat signals. ### Claim 19 (depends on 17) The non-transitory computer readable storage medium of claim 17, the program further executable to transmit version tags from the node, wherein the version tags correspond to a version level of a function supported by the node. ### Claim 20 (depends on 19) The non-transitory computer readable storage medium of claim 19, wherein the version tags correspond to a message format compatible with the version level of the function.