# US 9,438,466 B1 · Migrating virtual machines between oversubscribed and undersubscribed compute devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 9,438,466 B1
- Document ID: US9438466B1
- Status: Granted
- Kind: B1
- Application number: 13/524,333
- Filed: 2012-06-15
- Granted: 2016-09-06
- Earliest claimed: 2012-06-15
- Assignees: Juniper Networks, Inc.
- Applicants: Michael O'Gorman; Shesha Sreenivasamurthy
- Inventors: Michael O'Gorman; Shesha Sreenivasamurthy
- Web page: https://patentgrep.com/patent/US9438466B1

## CPC

- H04L 41/0897: by horizontal or vertical scaling of resources, or by migrating entities, e.g. virtual resources or entities
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 9/4856: {resumption being on a different machine, e.g. task migration, virtual machine migration (G06F9/5088 takes precedence)}
- G06F 9/5088: {involving task migration}
- H04L 41/0213: Standardised network management protocols, e.g. simple network management protocol [SNMP]
- H04L 41/0823: characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability (for optimising operational conditions of wireless networks H04W24/02)
- H04L 41/22: comprising specially adapted graphical user interfaces [GUI]
- H04L 69/329: in the application layer [OSI layer 7]
- G06F 2009/4557: {Distribution of virtual machine instances; Migration and load balancing}
- G06F 2009/45575: {Starting, stopping, suspending or resuming virtual machine instances}
- G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances}
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

In some embodiments, a non-transitory processor-readable medium includes code to cause a processor to receive, at a management device, an instantiation request for a first virtual machine. The code includes code to cause the processor to identify a first compute device at a first time such that the first compute device is undersubscribed at the first time. The code includes code to cause the processor to send an instruction to instantiate the first virtual machine at the compute device, and receive a signal indicating that a boot process associated with the first virtual machine is complete and that the virtual machine is instantiated at the first compute device. The code includes code to cause the processor to send at a second time, a signal to migrate the first virtual machine from the first compute device to a second compute device in response to the boot process being complete.

## Claims

### Claim 1

A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code comprising code to cause the processor to:
receive an instantiation request for a first virtual machine;
identify a first compute device at a first time and in response to the instantiation request, the first compute device implementing an undersubscribed policy at the first time causing a number of all virtual machines provisioned at the first compute device to be no greater than a total number of central processing units at the first compute device;
send, after the first time and before a second time, an instruction to instantiate the first virtual machine at the first compute device;
receive a signal indicating that a boot process associated with the first virtual machine is complete and that the first virtual machine is instantiated at the first compute device; and
send, at the second time, a signal to migrate the first virtual machine from the first compute device to a second compute device implementing an oversubscribed policy in response to the boot process being complete, the second compute device executing a second virtual machine at the second time.

### Claim 2 (depends on 1)

The non-transitory processor-readable medium of claim 1, wherein the second compute device executes the first virtual machine and the second virtual machine after the second time.

### Claim 3 (depends on 1)

The non-transitory processor-readable medium of claim 1, wherein the code to cause the processor to receive the instantiation request includes code to cause the processor to receive the instantiation request at an orchestration module implemented at the second compute device.

### Claim 4 (depends on 1)

The non-transitory processor-readable medium of claim 1, wherein the first compute device and the second compute device are within a data center.

### Claim 5 (depends on 1)

The non-transitory processor-readable medium of claim 1, wherein the instantiation request is associated with rebooting the first virtual machine.

### Claim 6 (depends on 1)

The non-transitory processor-readable medium of claim 1, further comprising code to cause the processor to:
send, at a third time after the second time and in response to a prediction of a resource-intensive event, a signal to migrate the virtual machine from the second compute device to a third compute device, the third compute device implementing an undersubscribed policy at the third time.

### Claim 7

An apparatus, comprising:
an orchestration module implemented in at least one of a memory or a processing device, the orchestration module configured to be operatively coupled to a compute device that implements an undersubscribed policy causing a number of all virtual machines provisioned at the compute device to be no greater than a total number of central processing units at the compute device and a compute device that implements an oversubscribed policy,
the orchestration module configured to receive an instantiation request for a virtual machine, the orchestration module configured to send an instruction to instantiate the virtual machine at the compute device that implements the undersubscribed policy in response to the instantiation request, the orchestration module configured to receive a signal indicating that a boot process associated with the virtual machine is complete and that the virtual machine is instantiated at the compute device that implements the undersubscribed policy, the orchestration module configured to send a signal to migrate the virtual machine from the compute device that implements the undersubscribed policy to the compute device that implements the oversubscribed policy in response to the boot process being complete.

### Claim 8 (depends on 7)

The apparatus of claim 7, further comprising:
a prediction module configured to receive a signal associated with a status of the virtual machine at the compute device that implements the oversubscribed policy, the prediction module configured to predict, based on the status, a resource-intensive event, the orchestration module configured to send a signal to migrate the virtual machine from the compute device that implements the oversubscribed policy to the compute device that implements the undersubscribed policy prior to the resource-intensive event.

### Claim 9 (depends on 7)

The apparatus of claim 7, wherein the orchestration module is configured to determine that the compute device that implements the undersubscribed policy is not executing a virtual machine prior to sending the instruction.

### Claim 10 (depends on 7)

The apparatus of claim 7, wherein the orchestration module is implemented at the compute device that implements the oversubscribed policy.

### Claim 11 (depends on 7)

The apparatus of claim 7, wherein the orchestration module is configured to be operatively coupled to the compute device that implements the undersubscribed policy and the compute device that implements the oversubscribed policy via a network.

### Claim 12 (depends on 7)

The apparatus of claim 7, wherein the orchestration module, the compute device that implements the undersubscribed policy and the compute device that implements the undersubscribed policy are within a data center.

### Claim 13 (depends on 7)

The apparatus of claim 7, wherein the instantiation request is associated with rebooting the virtual machine.

### Claim 14

An apparatus, comprising:
a compute device having a first memory and a processor operatively coupled to the first memory,
the processor configured to receive, from an orchestration module implemented in at least one of a second memory or a processing device, an instantiation request associated with a virtual machine, the processor configured to execute a boot process associated with the virtual machine in response to the instantiation request when the compute device is in an undersubscribed configuration causing a number of all virtual machines provisioned at the compute device to be no greater than a total number of central processing units at the compute device, the processor configured to send an indication to the orchestration module that the boot process is complete and that the virtual machine is instantiated at the compute device, the processor configured to receive, from the orchestration module, a signal to migrate the virtual machine from the compute device to an oversubscribed compute device in response to the indication and when the compute device is in the undersubscribed configuration.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the processor is configured to send, to the oversubscribed compute device, data associated with the virtual machine in response to the signal to migrate.

### Claim 16 (depends on 14)

The apparatus of claim 14, wherein the processor is configured to receive the instantiation request in response to the orchestration module determining an availability of the compute device.

### Claim 17 (depends on 14)

The apparatus of claim 14, wherein the virtual machine is a first virtual machine, the processor is configured to receive, from the orchestration module, a signal associated with migration of a second virtual machine from the oversubscribed compute device to the compute device in response to a prediction of a resource-intensive event associated with the second virtual machine and when the compute device is in the undersubscribed configuration.

### Claim 18 (depends on 14)

The apparatus of claim 14, wherein the instantiation request is associated with rebooting the virtual machine.

### Claim 19 (depends on 14)

The apparatus of claim 14, wherein the orchestration module, the compute device and the oversubscribed compute device are within a data center.

### Claim 20 (depends on 14)

The apparatus of claim 14, wherein the processor is configured to be operatively coupled to the oversubscribed compute device and the orchestration module via a network.