# US 10,311,027 B2 · System for live-migration and automated recovery of applications in a distributed system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,311,027 B2 - Document ID: US10311027B2 - Status: Granted - Kind: B2 - Application number: 14/109,545 - Filed: 2013-12-17 - Published: 2014-04-17 - Granted: 2019-06-04 - Earliest claimed: 2011-09-23 - Assignees: Open Invention Network, LLC - Applicants: Open Invention Network, LLC - Inventors: Luke Marsden - Web page: https://patentgrep.com/patent/US10311027B2 ## CPC - H04L 67/1095: Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes - G06F 3/061: {Improving I/O performance} - G06F 3/0647: {Migration mechanisms} - G06F 3/065: {Replication mechanisms} - G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]} - G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals} - G06F 9/5061: {Partitioning or combining of resources} - G06F 16/184: {implemented as replicated file system} - H04L 67/148: Migration or transfer of sessions - H04L 69/40: for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection (management of faults, events, alarms or notifications in data switching networks H04L41/06) - G06F 2209/505: Clust - G06F 2209/508: Monitor - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method and apparatus for distribution of applications amongst a number of servers, ensuring that changes to application data on a master for that application are asynchronously replicated to a number of slaves for that application. Servers may be located in geographically diverse locations; the invention permits data replication over high-latency and lossy network connections and failure-tolerance under hardware and network failure conditions. Access to applications is mediated by a distributed protocol handler which allows any request for any application to be addressed to any server, and which, when working in tandem with the replication system, pauses connections momentarily to allow seamless, consistent live-migration of applications and their state between servers. Additionally, a system which controls the aforementioned live-migration based on dynamic measurement of load generated by each application and the topological preferences of each application, in order to automatically keep servers at an optimum utilization level. ## Claims ### Claim 1 Computer software which, when executed by appropriate processing means, causes the processing means to implement a method of managing snapshots of a filesystem, where the filesystem is replicated across multiple servers connected in a cluster comprising: indentifying each snapshot by a snapnode object in the form of a binary sequence comprising a snapshot identifier, a parent pointer to an earlier snapshot on a specific server where the snapshot was taken, and the set of servers where this snapshot is presently stored; storing a graph of snapnode objects of a set of snapshots of a filesystem on each of the multiple servers, one of the servers being an active master of the file system; the active master taking a new snapshot of the filesystem and creating a snapnode object for the new snapshot identifying the active master as a server where the new snapshot is stored; and transmitting the new snapshot to the other servers of the multiple servers; and modifying the snapnode object to identify the other servers as servers where the new snapshot is stored, wherein the method is used to manage recovery of a file system after an event in which the active master is to confirm or modify its status. ### Claim 2 (depends on 1) Computer software according to claim 1 wherein the event is the partitioning of a cluster of servers in which the active master and the other servers are connected, wherein after recovery from the partition there are at least two candidate master servers, each with a graph of snapnode objects for the filesystem, wherein the graph at each candidate master is traversed to assess its value, and the candidate master with the graph indicating a highest value adopts a role as new master for the filesystem. ### Claim 3 (depends on 2) Computer software according to claim 2 wherein prior to executing the comparison, the snapshot data is globally synchronised across the servers, whereby divergence of a version of the data at each candidate master can be assessed against the globally synchronised snapshot data. ### Claim 4 (depends on 3) Computer software according to claim 3 wherein the event is loss of at least one other server of the multiple servers, which was acting as a slave server to the active master, wherein after a replacement slave has been designated by a master, the master instructs the new slave to replicate a complete copy of the filesystem so that replication can begin from a current point. ### Claim 5 (depends on 2) Computer software according to claim 2, the implemented method comprising the step of saving snapshots from a given slice point in the graph to a local storage area. ### Claim 6 (depends on 2) Computer software according to claim 2, the implemented method comprising the step of pruning a snapshot. ### Claim 7 (depends on 2) Computer software according to claim 2, implemented method comprising the step of determining which action to take in order to resolve a divergence of graphs on multiple servers representing the same filesystem based on: (1) a current master for the filesystem; (2) a graph of snapnode objects for that filesystems global state; (3) a list of current slave servers to that master for that filesystem. ### Claim 8 (depends on 1) Computer software according to claim 1, wherein the method further comprises: detecting an event selected from the following: (i) failure of current master server for the application, (ii) a partitioning of the cluster, (iii) a reduction in the number of servers in the cluster, (iv) an increase in the number of servers in the cluster, (v) introduction into the cluster of a server for which a user has expressed a preference for hosting the application, (vi) a change in load of applications amongst servers in the cluster, such that a load rebalancing event is required; and in response to the event determining a new one of the servers being the active master of the filesystem. ### Claim 9 A method of managing snapshots of a filesystem, where the filesystem is replicated across multiple servers connected in a cluster comprising: indentifying each snapshot by a snapnode object in the form of a binary sequence comprising a snapshot identifier, a parent pointer to an earlier snapshot on a specific server where the snapshot was taken, and the set of servers where this snapshot is presently stored; storing a graph of snapnode objects of a set of snapshots of a filesystem on each of the multiple servers, one of the servers being an active master of the file system; the active master taking a new snapshot of the filesystem and creating a snapnode object for the new snapshot identifying the active master as a server where the new snapshot is stored; and transmitting the new snapshot to the other servers of the multiple servers; and modifying the snapnode object to identify the other servers as servers where the new snapshot is stored, wherein the method is used to manage recovery of a file system after an event in which the active master is to confirm or modify its status. ### Claim 10 (depends on 9) A method according to claim 9 wherein the event is the partitioning of a cluster of servers in which the active master and the other servers are connected, wherein after recovery from the partition there are at least two candidate master servers, each with a graph of snapnode objects for the filesystem, wherein the graph at each candidate master is traversed to assess its value, and the candidate master with the graph indicating a highest value adopts a role as new master for the filesystem. ### Claim 11 (depends on 10) A method according to claim 10 wherein prior to executing the comparison, the snapshot data is globally synchronised across the servers, whereby divergence of a version of the data at each candidate master can be assessed against the globally synchronised snapshot data. ### Claim 12 (depends on 9) A method according to claim 9 wherein the event is loss of at least one other server of the multiple servers, which was acting as a slave server to the active master, wherein after a replacement slave has been designated by a master, the master instructs the new slave to replicate a complete copy of the filesystem so that replication can begin from a current point. ### Claim 13 (depends on 9) A method according to claim 9 comprising the step of saving snapshots from a given slice point in the graph to a local storage area. ### Claim 14 (depends on 9) A method according to claim 9 comprising the step of pruning a snapshot. ### Claim 15 (depends on 9) A method according to claim 9 comprising the step of determining which action to take in order to resolve a divergence of graphs on multiple servers representing the same filesystem based on: (4) a current master for the filesystem; (5) a graph of snapnode objects for that filesystems global state; (6) a list of current slave servers to that master for that filesystem. ### Claim 16 (depends on 9) A method according to claim 9 wherein the snapshot identifier identifies the time at which the snapshot was taken and the server on which the snapshot was taken. ### Claim 17 (depends on 9) A method or server according to claim 9 wherein a user interface is presented to a user for permitting access via a user to snapshots selected by a user. ### Claim 18 (depends on 9) A method according to claim 9 further comprising storing snapshots taken of the filesystem such that a previous snapshot of the filesystem can be selected by a user from the stored snapshots to restore the system to its state at the time of the selected snapshot.