# US 11,112,975 B2 · Background migration of virtual storage

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,112,975 B2
- Document ID: US11112975B2
- Status: Granted
- Kind: B2
- Application number: 16/020,820
- Filed: 2018-06-27
- Published: 2019-04-18
- Granted: 2021-09-07
- Earliest claimed: 2010-02-08
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Dustin L. Green; Jacob K. Oshins; Lars Reuther
- Web page: https://patentgrep.com/patent/US11112975B2

## CPC

- G06F 3/0619: {in relation to data integrity, e.g. data losses, bit errors}
- G06F 3/0617: {in relation to availability}
- G06F 3/0647: {Migration mechanisms}
- G06F 3/065: {Replication mechanisms}
- G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling}
- G06F 3/0665: {at area level, e.g. provisioning of virtual or logical volumes}
- G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}
- G06F 3/0689: {Disk arrays, e.g. RAID, JBOD}
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 11/2066: {Optimisation of the communication load}
- G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Described is a technology by which a virtual hard disk is migrated from a source storage location to a target storage location without needing any shared physical storage, in which a machine may continue to use the virtual hard disk during migration. This facilitates use the virtual hard disk in conjunction with live-migrating a virtual machine. Virtual hard disk migration may occur fully before or after the virtual machine is migrated to the target host, or partially before and partially after virtual machine migration. Background copying, sending of write-through data, and/or servicing read requests may be used in the migration. Also described is throttling data writes and/or data communication to manage the migration of the virtual hard disk.

## Claims

### Claim 1

In a computing environment, a method performed on at least one processor, comprising:
executing control, by a target host, of a virtual machine executing at a source host before migration of virtual disk data stored in a storage component of the virtual machine;
migrating the virtual machine and the storage component of the virtual machine from a first storage unit of the source host to a second storage unit of the target host; and
servicing read/write requests during the migration of the virtual machine and the storage component of the virtual machine, including:
obtaining the virtual disk data via background copying of inbound data at an inbound rate based on an inbound transfer rate threshold;
throttling the background copying to utilize less than a percentage of a line rate;
obtaining on demand reads at a higher priority than the background copying of the virtual disk data;
obtaining newly-written data at a higher priority than the background copying of the virtual disk data;
throttling the obtained newly-written data based at least on a rule that at least some of the background copying proceeds while the obtained newly-written data proceeds; and
throttling a transfer rate of the inbound data during the migration to stay below a limit specified by the target host.

### Claim 2 (depends on 1)

The method of claim 1, wherein the throttling the transfer rate of inbound data during the migration based on limits specified by the target host is achieved through packet prioritization.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
throttling a transfer rate of newly-written data during the migration based on network transmission capabilities.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
obtaining, from the source host, one or more descriptions of virtual disk data regions available on at least one other source.

### Claim 5 (depends on 4)

The method of claim 4, further comprising:
caching the obtained one or more descriptions to avoid querying the source host for data in the virtual disk data regions available on the at least one other source.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
delaying the background copying for an indefinite period prior to starting,
wherein the line rate is a maximum transfer rate of a network.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first storage unit of the source host and the second storage unit of the target host are not shared between the source host and the target host.

### Claim 8 (depends on 1)

The method of claim 1, wherein the inbound transfer rate threshold is based at least on a write capacity of the target host.

### Claim 9

One or more computer storage medias that are not a signal per se having computer-executable instructions that, upon execution by a computer cause the computer to perform operations comprising:
executing control, by a target host, of a virtual machine executing at a source host before migration of virtual disk data stored in a storage component of the virtual machine;
migrating the virtual machine and the storage component of the virtual machine from a first storage unit of the source host to a second storage unit of the target host;
obtaining, by the target host, the virtual disk data via background copying of inbound data at a rate based on an inbound transfer rate threshold;
throttling the background copying to utilize less than a percentage of a line rate;
obtaining, by the target host, reads on demand as requested at a higher priority than the background copying of the virtual disk data;
obtaining, by the target host, newly-written data at a higher priority than the background copying of the virtual disk data;
servicing, by the target host, read/write requests during the migration of the virtual machine and the storage component of the virtual machine; and
throttling a transfer rate of the inbound data during the migration to stay below a limit specified by the target host.

### Claim 10 (depends on 9)

The one or more computer storage medias that are not a signal per se of claim 9, wherein the throttling the transfer rate of inbound data during the migration based on limits specified by the target host is achieved through packet prioritization.

### Claim 11 (depends on 9)

The one or more computer storage medias that are not a signal per se of claim 9, having further computer-executable instructions that, upon execution by a computer cause the computer to perform operations comprising:
throttling a transfer rate of newly-written data during the migration based on network transmission capabilities.

### Claim 12 (depends on 9)

The one or more computer storage medias that are not a signal per se of claim 9, having further computer-executable instructions that, upon execution by a computer cause the computer to perform operations comprising:
obtaining, from the source host, one or more descriptions of virtual disk data regions available on at least one other source.

### Claim 13 (depends on 12)

The one or more computer storage medias that are not a signal per se of claim 12, having further computer-executable instructions that, upon execution by a computer cause the computer to perform operations comprising:
caching the obtained descriptions to avoid querying the source host for data in the virtual disk data regions available on the at least one other source.

### Claim 14 (depends on 9)

The one or more computer storage medias that are not a signal per se of claim 9, having further computer-executable instructions that, upon execution by a computer cause the computer to perform operations comprising:
throttling the background copying to utilize less than a maximum transfer rate of a network.

### Claim 15 (depends on 9)

The one or more computer storage medias that are not a signal per se of claim 9, wherein the first storage unit of the source host and the second storage unit of the target host are not shared between the source host and the target host.

### Claim 16

In a computing environment, a system comprising:
a memory storing data associated with one or more virtual machines; and
a target host configured to:
execute control of a virtual machine executing at a source host before migration of virtual disk data stored in a storage component of the virtual machine;
migrate the virtual machine and the storage component of the virtual machine from a first storage unit of the source host to a second storage unit of the target host; and
service read/write requests during the migration of the virtual machine and the storage component of the virtual machine, including by:
obtaining the virtual disk data via background copying of inbound data at a rate based on an inbound network data volume threshold;
throttling the background copying to utilize less than a percentage of a line rate;
throttling a transfer rate of the inbound data during the migration to stay below limits specified by the target host;
obtaining on demand reads at a higher priority than the background copying of the virtual disk data; and
obtaining newly-written data at a higher priority than the background copying of the virtual disk data; and
throttling the obtained newly-written data based at least on a rule that at least some of the background copying proceeds while the obtained newly-written data proceeds.

### Claim 17 (depends on 16)

The system of claim 16, wherein the target host is further configured to obtain, from the source host, one or more descriptions of virtual disk data regions available on at least one other source.

### Claim 18 (depends on 17)

The system of claim 17, wherein the target host is further configured to cache the obtained one or more descriptions to avoid querying the source host for data in the virtual disk data regions available on the at least one other source.

### Claim 19 (depends on 16)

The system of claim 16, wherein the target host is further configured to throttle the background copying to utilize less than a maximum transfer rate of a network.

### Claim 20 (depends on 16)

The system of claim 16, wherein the first storage unit of the source host and the second storage unit of the target host are not shared between the source host and the target host.