Method and system for enabling checkpointing fault tolerance across remote virtual machines

#As large-scale enterprises continue to adopt virtualization platforms as the foundation of their data centers, virtual machine (VM) fault tolerance has become an increasingly important feature to be provided by virtualization platform providers. Because a single host server in a virtualized data center can support multiple VMs, failure of that host server can bring down a multitude of services that were provided by the different VMs running on the failed host server. As such, virtualization platforms need to provide a mechanism to quickly resurrect a failed VM on a different host server so that the enterprise can maintain the quality of its service.

#Currently, providing fault tolerance for a primary VM is typically achieved by providing a backup VM that runs on a server residing in a different “fault domain” from the server of the primary virtual machine. A fault domain can generally be described as a set of host servers in a data center (or data centers) that share a number of specified attributes and/or characteristics that results in a higher probability of failure of host servers in the fault domain upon a failure of one of the host servers in the fault domain. The attributes and/or characteristics utilized by an enterprise to define its data center fault domains depend upon the type of disasters and the level of recovery that the enterprises desire to achieve. For example, an enterprise may choose to define its fault domains based upon the physical proximity of host servers (storage rack location, geographic locations, etc.), the dependency of such servers on shared hardware (networked storage, power sources, physical connections, etc.) or software technologies (shared file systems, etc.), and the like. A well-constructed fault domain minimizes the correlation of a failure of a VM in one fault domain with the failure of another VM in a different fault domain.

#VM fault tolerance may be provided using deterministic replay, checkpointing, or a hybrid of the two, which is disclosed in U.S. patent application Ser. No. 12/259,762, filed on Aug. 28, 2008, the entire contents of which are incorporated by reference herein. With replay techniques, essential portions of a primary VM's instruction stream (e.g., non-deterministic events within the primary VM's instruction stream) are captured in real-time (e.g., by a hypervisor layer or virtual machine monitor component of the primary VM) and transmitted to a backup VM (e.g., presumably located in a different fault domain) to “replay” the primary VM's execution in a synchronized fashion. If the primary VM fails, the backup VM can then take over without discernable loss of time. While replay techniques provide a robust fault tolerance solution with fast recovery times, they are less viable, for example, when non-deterministic events become more frequent or more difficult to identify within instruction streams, as is the case with virtual machines that support SMP (symmetric multiprocessing) architectures with multiple virtual CPUs.

#FIG. 1A depicts a block diagram of an embodiment of a network architecture for a primary and backup virtual machine using an intermediary computer system.

#FIG. 1B depicts a block diagram of a second embodiment of a network architecture for a primary and backup virtual machine using an intermediary computer system.

#FIG. 2 depicts a flow diagram for transmitting an updated state of a primary virtual machine to a backup virtual machine using an intermediary computer system.

#FIG. 1A depicts a block diagram of an embodiment of a network architecture for a primary and backup virtual machine using an intermediary computer system. A primary VM 100 resides on a primary server 105 in a fault domain 110 of a data center. One example of a primary server 105 that supports virtual machines is a server that runs the VMware ESX® hypervisor product, which is commercially available from VMware, Inc. of Palo Alto, Calif. (although it should be recognized that any virtualization technologies may be used consistent with the teachings herein, including Xen®, Microsoft Hyper-V and the like).

#In the description that follows, the focus will be upon the embodiment in which a bandwidth concern is at least one motivation for configuring an intermediary computer system 115 to preferentially pass less recently updated memory pages. However, there are motivations for utilizing the method even where bandwidth limitations is not an issue.

#An intermediary computer system 115 (hereinafter, “intermediary”) is situated in close enough proximity to server 105 such that a high bandwidth connection 120 can be placed between server 105 and intermediary 115. High bandwidth connection 120, as described further below, provides sufficient bandwidth capacity to support the transmission of checkpoint information between primary VM 100 and intermediary 115 during primary VM's 100 execution. For example, in one embodiment, high bandwidth connection 120 provides sufficient bandwidth capacity for intermediary 115 to efficiently receive checkpoint information from the primary VM 100 at a rate of 50 to 100 times per second, with each set of checkpoint information potentially comprising multiple gigabytes of data. Although intermediary 115 is located at a sufficient physical proximity to primary VM 100 to enable high bandwidth connection 120, intermediary 115 is also located in a different fault domain 125 than fault domain 110 of primary VM 100, such that a failure of primary VM 100 (or server 105) is not correlated to (or otherwise does not have a significant correlation to) a failure of intermediary 115. As further depicted in FIG. 1A, a backup VM 130 resides on a backup server 135 that shares fault domain 125 with intermediary 115. A lower bandwidth connection 140 between intermediary 115 and backup VM 130 provides flexibility to situate backup VM 130 in a location geographically remote from intermediary 115 and/or primary VM 100.