# US 10,481,940 B2 · Post-copy based live virtual machine migration via speculative execution and pre-paging > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,481,940 B2 - Document ID: US10481940B2 - Status: Granted - Kind: B2 - Application number: 15/849,505 - Filed: 2017-12-20 - Published: 2018-05-17 - Granted: 2019-11-19 - Earliest claimed: 2016-06-10 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Venkatesh Srinivas - Web page: https://patentgrep.com/patent/US10481940B2 ## CPC - G06F 9/45558: {Hypervisor-specific management and integration aspects} - G06F 2009/4557: {Distribution of virtual machine instances; Migration and load balancing} - G06F 2009/45583: {Memory management, e.g. access or allocation} - G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The method includes post copy migrating a guest virtual machine from a migration source host to a migration target host, executing the instruction stream of the guest virtual machine on the migration target host, and determining, by the migration target host, when the guest virtual machine encounters the page fault. When the guest virtual machine encounters the page fault, the method includes requesting, by the migration target host, the not-present page from the migration source host for loading on the migration target host, and emulating, on an emulator executing on or in communication with the migration target host, continued execution of the instruction stream of the guest virtual machine. The method also includes identifying, by the emulator, future page references to future pages from the emulated continued execution of the instruction stream, and requesting the future pages from the migration source host in parallel with the not-present page request. ## Claims ### Claim 1 A method comprising: determining, by a migration target host, that a guest virtual machine being post-copy migrated from a migration source host to the migration target host and executed on the migration target host encounters a page fault for a page reference in an instruction stream of the guest virtual machine to a not-present page; requesting, by the migration target host, the not-present page from the migration source host for loading on the migration target host; emulating, on an emulator executing on or in communication with the migration target host, continued execution of the instruction stream of the guest virtual machine; identifying, by the emulator, future page references to future pages from the emulated continued execution of the instruction stream; examining, by the emulator, at least one instruction in the instruction stream associated with one of the identified future page references to identify at least one additional future page reference; and requesting, by the migration target host, the future pages corresponding to the identified future page references and the at least one identified additional future page reference from the migration source host and the not-present page request for the not-present page. ### Claim 2 (depends on 1) The method of claim 1, wherein requesting the future pages corresponding to the identified future page references from the migration source host is in parallel with requesting the not-present page request for the not-present page, so as to prefetch the future pages to the migration target host while loading the not-present page on the migration target host. ### Claim 3 (depends on 1) The method of claim 1, further comprising, while emulating the continued execution of the instruction stream of the guest virtual machine, setting aside instructions that are dependent on loaded values of the not-present page, allowing the emulator to identify future page references beyond future page faults. ### Claim 4 (depends on 1) The method of claim 1, wherein the migration target host comprises: one or more computing devices; and one or more memory hosts in communication with the one or more computing devices, the one or more memory hosts storing pages. ### Claim 5 (depends on 1) The method of claim 1, further comprising identifying the future page references based on a relationship between the future page references and the not-present page. ### Claim 6 (depends on 5) The method of claim 5, where the relationship comprises a page navigation map between a parent page and one or more child pages. ### Claim 7 (depends on 1) The method of claim 1, further comprising ceasing emulation of the continued execution of the instruction stream by the emulator when the not-present page has been delivered to the migration target host. ### Claim 8 (depends on 7) The method of claim 7, further comprising continuing execution of the instruction stream of the guest virtual machine on the migration target host when the not-present page is resident on the migration target host. ### Claim 9 (depends on 1) The method of claim 1, wherein identifying future page references from the emulated continued execution of the instruction stream further comprises identifying a future page address of the identified future page references. ### Claim 10 A migration target host comprising: data processing hardware; and memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising: determining that a guest virtual machine being post-copy migrated from a migration source host to the memory hardware and executed on the data processing hardware encounters a page fault for a page reference in an instruction stream of the guest virtual machine to a not-present page; requesting the not-present page from the migration source host for loading on the memory hardware; emulating, on an emulator executing on the data processing hardware or in communication with the migration target host, continued execution of the instruction stream of the guest virtual machine; identifying, by the emulator, future page references to future pages from the emulated continued execution of the instruction stream; examining, by the emulator, at least one instruction in the instruction stream associated with one of the identified future page references to identify at least one additional future page reference; and requesting the future pages corresponding to the identified future page references and the at least one identified additional future page reference from the migration source host and the not-present page request for the not-present page. ### Claim 11 (depends on 10) The migration target host of claim 10, wherein requesting the future pages corresponding to the identified future page references from the migration source host is in parallel with requesting the not-present page request for the not-present page, so as to prefetch the future pages to the migration target host while loading the not-present page on the migration target host. ### Claim 12 (depends on 10) The migration target host of claim 10, wherein the operations further comprise, while emulating the continued execution of the instruction stream of the guest virtual machine, setting aside instructions that are dependent on loaded values of the not-present page, allowing the emulator to identify future page references beyond future page faults. ### Claim 13 (depends on 10) The migration target host of claim 10, wherein the operations further comprise requesting the identified future page references from the migration source host as the future page references are identified. ### Claim 14 (depends on 10) The migration target host of claim 10, wherein the operations further comprise identifying the future page references based on a relationship between the future page reference and the not-present page. ### Claim 15 (depends on 14) The migration target host of claim 14, where the relationship comprises a page navigation map between a parent page and one or more child pages. ### Claim 16 (depends on 10) The migration target host of claim 10, wherein the operations further comprise ceasing emulation of the continued execution of the instruction stream by the emulator when the not-present page has been delivered to the memory hardware of the migration target host. ### Claim 17 (depends on 16) The migration target host of claim 16, wherein the operations further comprise continuing execution of the instruction stream of the guest virtual machine when the not-present page is resident on the memory hardware of the migration target host. ### Claim 18 (depends on 10) The migration target host of claim 10, wherein identifying future page references from the emulated continued execution of the instruction stream further comprises identifying a future page address of the identified future page references.