# US 10,771,534 B2 · Post data synchronization for domain migration > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,771,534 B2 - Document ID: US10771534B2 - Status: Granted - Kind: B2 - Application number: 16/297,065 - Filed: 2019-03-08 - Published: 2019-07-04 - Granted: 2020-09-08 - Earliest claimed: 2015-12-28 - Assignees: Amazon Technologies, Inc. - Applicants: Amazon Technologies, Inc. - Inventors: Alexey Gadalin; Nikolay Krasilnikov; Rudresh Amin; Weili Zhong McClenahan; Anton Valter - Web page: https://patentgrep.com/patent/US10771534B2 ## CPC - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - G06F 9/45558: {Hypervisor-specific management and integration aspects} - H04L 67/1095: Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - H04L 67/60: Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources (admission control or resource allocation H04L47/70) - G06F 2009/4557: {Distribution of virtual machine instances; Migration and load balancing} - G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage} - G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances} - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Methods and apparatus for post data synchronization in live migration of domains. Host devices on a network may implement virtual machines (VMs) as domains in an execution environment, and may provide local persistent storage for data of the VMs. A migration technique for moving a domain including the persistent data from one host device to another host device is described in which the VM is instantiated on the target device, and the domain is switched to the target device. Synchronization of the VM's data from the persistent storage on the source device to the target device is then initiated, for example according to a distributed replicated storage technique that makes the target device's persistent storage the primary storage and the source device's persistent storage the secondary storage for the VM. Once the data is synchronized, the VM and its respective storage on the source device are released. ## Claims ### Claim 1 A system, comprising: a target host device, comprising one or more processors and local persistent storage, configured to implement one or more virtual machines (VMs), wherein the local persistent storage of the target host device is allocated to the one or more VMs, wherein the target host device is part of a provider network that comprises a plurality of host devices, including a source host device, each host device implementing one or more VMs in an execution environment of the respective host device; wherein the one or more VMs on the target host device comprises a migrating VM, wherein the migrating VM is being migrated from the source host device; and wherein the target host device is configured to: obtain a read request directed to the migrating VM on the target host device; attempt to retrieve data associated with the read request from the local persistent storage of the target host device; determine that at least a portion of the data associated with the read request does not reside in the local persistent storage of the target host device; communicate with the source host device to retrieve at least the portion of the data associated with the read request from the local persistent storage of the source host device; and respond to the read request using at least the data portion from the local persistent storage of the source host device. ### Claim 2 (depends on 1) The system of claim 1, wherein the target host device is further configured to: store at least the portion of the data associated with the read request retrieved from the local persistent storage of the source host device in the local persistent storage of the target host device. ### Claim 3 (depends on 1) The system as recited in claim 1, wherein the target host device comprises a distributed replicated storage module, wherein said attempting, determining, and communicating is performed by the distributed replicated storage module of the target host device, and wherein to communicate with the source host device, the distributed replicated storage module of the target host device is further configured to: communicate with a distributed replicated storage module of the source host device. ### Claim 4 (depends on 1) The system as recited in claim 1 wherein, prior to obtaining the read request directed to the migrating VM on the target host device, the target host device is further configured to: launch the migrating VM based on a machine image of the VM in the execution environment of the target host device. ### Claim 5 (depends on 1) The system as recited in claim 1 wherein, prior to obtaining the read request directed to the migrating VM on the target host device, the target host device is further configured to: establish a communications link over the provider network with the source host device; and initiate a synchronization of at least some data from the local persistent storage of the source host device to the local persistent storage of the target host device over the communication link. ### Claim 6 (depends on 5) The system as recited in claim 5, wherein said initiating a synchronization comprises launching a background process to copy the least some data from the local persistent storage of the source host device to the local persistent storage of the target host device over the communications link of the provider network. ### Claim 7 (depends on 1) The system of claim 1, wherein the target host device is further configured to: obtain a second read request directed to the migrating VM on the target host device; attempt to retrieve data associated with the second read request from the local persistent storage of the target host device; determine that the data associated with the second read request resides in the local persistent storage of the target host device; and respond to the second read request using at least the data associated with the second read request residing in the local persistent storage of the target host device. ### Claim 8 (depends on 1) The system of claim 1, wherein the target host device is further configured to: obtain a write request specifying data to be persisted for the migrating VM; store the data specified by the write request to the local persistent storage of the target host device; and communicate with the source host device to copy the data specified by the write request to the source host device for storage to the local persistent storage of the source host device. ### Claim 9 A method, performed by a target host device of a provider network, comprising: obtaining, by the target host device of the provider network, a read request directed to a migrating VM on the target host device, wherein the migrating VM on the target host device is being migrated from a source host device of the provider network; attempting to retrieve data associated with the read request from a local persistent storage of the target host device; determining that at least a portion of the data associated with the read request does not reside in the local persistent storage of the target host device; communicating with the source host device to retrieve at least the portion of the data associated with the read request from a local persistent storage of the source host device; and responding to the read request using at least the data portion from the local persistent storage of the source host device. ### Claim 10 (depends on 9) The method of claim 9, further comprising: storing at least the portion of the data associated with the read request retrieved from the local persistent storage of the source host device in the local persistent storage of the target host device. ### Claim 11 (depends on 9) The method of claim 9, wherein prior to obtaining the read request directed to the migrating VM on the target host device, the method further comprises: switching a domain corresponding to the migrating VM from the source host device to the target host device. ### Claim 12 (depends on 9) The method as recited in claim 9 wherein prior to obtaining the read request directed to the migrating VM on the target host device, the method further comprises: launching the migrating VM on the target host device based on a machine image of the VM in the execution environment of the target host device. ### Claim 13 (depends on 9) The method as recited in claim 9 wherein prior to obtaining the read request directed to the migrating VM on the target host device, the method further comprises: establishing a communications link over the provider network with the source host device; and initiating a synchronization of at least some data from the local persistent storage of the source host device to the local persistent storage of the target host device over the communication link. ### Claim 14 (depends on 13) The method as recited in claim 13, wherein said initiating a synchronization further comprises: launching a background process to copy the least some data from the local persistent storage of the source host device to the local persistent storage of the target host device over the communications link of the provider network. ### Claim 15 (depends on 9) The method of claim 9, further comprising: obtaining a second read request directed to the migrating VM on the target host device; attempting to retrieve data associated with the second read request from the local persistent storage of the target host device; determining that the data associated with the second read request resides in the local persistent storage of the target host device; and responding to the second read request using at least the data associated with the second read request residing in the local persistent storage of the target host device. ### Claim 16 (depends on 9) The method of claim 9, further comprising: obtaining a write request specifying data to be persisted for the migrating VM; storing the data specified by the write request to the local persistent storage of the target host device; and communicating with the source host device to copy the data specified by the write request to the source host device for storage to the local persistent storage of the source host device. ### Claim 17 One or more non-transitory computer-readable storage media storing program instructions that when executed on or across one or more processors cause the one or more processors to implement a target host device on a provider network to: obtain a read request directed to a migrating VM on the target host device, wherein the migrating VM on the target host device is being migrated from a source host device of the provider network; attempt to retrieve data associated with the read request from a local persistent storage of the target host device; determine that at least a portion of the data associated with the read request does not reside in the local persistent storage of the target host device; communicate with the source host device to retrieve at least the portion of the data associated with the read request from a local persistent storage of the source host device; and respond to the read request using at least the data portion from the local persistent storage of the source host device. ### Claim 18 (depends on 17) The one or more non-transitory computer-readable storage media of claim 17, wherein the program instructions further cause the one or more processors to: store at least the portion of the data associated with the read request retrieved from the local persistent storage of the source host device in the local persistent storage of the target host device. ### Claim 19 (depends on 17) The one or more non-transitory computer-readable storage media of claim 17, wherein the program instructions further cause the one or more processors to: obtain a second read request directed to the migrating VM on the target host device; attempt to retrieve data associated with the second read request from the local persistent storage of the target host device; determine that the data associated with the second read request resides in the local persistent storage of the target host device; and respond to the second read request using at least the data associated with the second read request residing in the local persistent storage of the target host device. ### Claim 20 (depends on 17) The one or more non-transitory computer-readable storage media of claim 17, wherein the program instructions further cause the one or more processors to: obtain a write request specifying data to be persisted for the migrating VM; store the data specified by the write request to the local persistent storage of the target host device; and communicate with the source host device to copy the data specified by the write request to the source host device for storage to the local persistent storage of the source host device.