# US 9,436,751 B1 · System and method for live migration of guest > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,751 B1 - Document ID: US9436751B1 - Status: Granted - Kind: B1 - Application number: 14/132,477 - Filed: 2013-12-18 - Granted: 2016-09-06 - Earliest claimed: 2013-12-18 - Assignees: Google Inc. - Applicants: Google Inc. - Inventors: Benjamin Charles Serebrin - Web page: https://patentgrep.com/patent/US9436751B1 ## CPC - G06F 9/4856: {resumption being on a different machine, e.g. task migration, virtual machine migration (G06F9/5088 takes precedence)} - G06F 9/45558: {Hypervisor-specific management and integration aspects} - G06F 2009/4557: {Distribution of virtual machine instances; Migration and load balancing} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method and apparatus are provided to perform live migration of a guest in a computer system using device assignment. In this method and apparatus, one or more pages of the guest are copied to a target computer system. It is determined which pages have been copied, and what incremental changes have been made to the pages that were copied. For example, the incremental changes may be indicated to a hypervisor of an initial host of the guest by a network interface controller or other device in the computer system. The incremental changes are then copied to the target computer system. Detection and copying of incremental changes may continue until a time when all dirty pages can be copied to the target computer system. ## Claims ### Claim 1 A method for live migration of a guest in a computer system using device assignment, the method comprising: copying one or more pages of the guest from an initial computer system to a target computer system, the initial computer system having one or more processors coupled to a network interface controller (NIC) through an input/output memory management unit (IOMMU); determining, using the one or more processors, which pages have been copied; determining, using the one or more processors, incremental changes made to the pages that have been copied, the determining including accessing page table entries in the IOMMU, wherein the page table entries were modified by the NIC; and copying the incremental changes from the initial computer system to the target computer system. ### Claim 2 (depends on 1) The method of claim 1, wherein accessing information provided by the NIC comprises accessing a list of dirtied pages tracked by the NIC. ### Claim 3 (depends on 1) The method of claim 1, wherein accessing information provided by the NIC comprises accessing a buffer of bytes, the buffer comprising a plurality of entries with each entry corresponding to at least a portion of a page of the guest, wherein the buffer is updated by the NIC when the NIC writes to the page of the guest. ### Claim 4 (depends on 1) The method of claim 1, wherein determining the incremental changes comprises receiving information from the guest indicating the pages for which the guest provided write access. ### Claim 5 (depends on 1) The method of claim 1, wherein determining which pages have been copied comprises marking the copied one or more pages of the guest as one of not dirty and not writeable. ### Claim 6 (depends on 1) The method of claim 1, further comprising determining whether a rate at which pages are dirtied approximates a rate at which dirtied pages can be copied to the target computer system. ### Claim 7 (depends on 6) The method of claim 6, further comprising continuing to determine and copy incremental changes if the rate at which pages are dirtied does not approximate a rate at which dirtied pages can be copied. ### Claim 8 (depends on 6) The method of claim 6, further comprising copying all dirtied pages and resuming processing on the target computer device if the rate at which pages are dirtied approximates a rate at which dirtied pages can be copied. ### Claim 9 An apparatus for live migration of a guest in a computer system using device assignment, the apparatus comprising: a memory; one or more processors in communication with the memory, the one or more processors coupled to a network interface controller (NIC) through an input/output memory management unit (IOMMU), the one or more processors performing tasks comprising: copying one or more pages of the guest to a target computer system; determining which pages have been copied; determining incremental changes made to the pages that have been copied, the determining including accessing a buffer of bytes, the buffer comprising a plurality of entries with each entry corresponding to at least a portion of a page of the guest, wherein the buffer is updated by the NIC when the NIC writes to the page of the guest; and copying the incremental changes to the target computer system. ### Claim 10 (depends on 9) The apparatus of claim 9, wherein accessing information provided by the NIC comprises accessing a list of dirtied pages tracked by the NIC. ### Claim 11 (depends on 9) The apparatus of claim 9, wherein accessing information provided by the NIC comprises accessing page table entries in the input/output memory management unit, wherein the page table entries were modified by the NIC. ### Claim 12 (depends on 9) The apparatus of claim 9, wherein determining the incremental changes comprises receiving information from the guest indicating the pages for which the guest provided write access. ### Claim 13 (depends on 9) The apparatus of claim 9, further comprising determining whether a rate at which pages are dirtied approximates a rate at which dirtied pages can be copied to the target computer system. ### Claim 14 (depends on 13) The apparatus of claim 13, further comprising copying all dirtied pages and resuming processing on the target computer system if the rate at which pages are dirtied approximates a rate at which dirtied pages can be copied. ### Claim 15 A method for live migration of a guest in a computer system using device assignment, the method comprising: copying one or more pages of the guest from an initial computer system to a target computer system, the initial computer system having one or more processors coupled to a network interface controller (NIC) through an input/output memory management unit (IOMMU); receiving at the one or more processors information from the guest regarding which of the one or more pages it intends to give write access to the NIC; determining, using the one or more processors, which pages have been copied; determining, by the NIC, incremental changes made to the pages that have been copied based on the information received from the guest; and copying, using the one or more processors, the incremental changes from the initial computer system to the target computer system. ### Claim 16 A system for live migration of a guest in a computer system using device assignment, comprising: a memory; one or more processors in communication with the memory, the one or more processors coupled to a network interface controller (NIC) through an input/output memory management unit (IOMMU), the one or more processors performing tasks comprising: copying one or more pages of the guest to a target computer system; determining which pages have been copied; determining incremental changes made to the pages that have been copied, the determining including accessing page table entries in the IOMMU, wherein the page table entries were modified by the NIC; and copying the incremental changes to the target computer system. ### Claim 17 A system for live migration of a guest in a computer system using device assignment, comprising: a memory; one or more processors in communication with the memory, the one or more processors coupled to a network interface controller (NIC) through an input/output memory management unit (IOMMU), the one or more processors performing tasks comprising: copying one or more pages of the guest to a target computer system; receiving information from the guest regarding which of the one or more pages it intends to give write access to the NIC; determining which pages have been copied; determining, with the NIC, incremental changes made to the pages that have been copied based on the information received from the guest; and copying, using the one or more processors, the incremental changes to the target computer system.