# US 8,543,790 B2 · System and method for cooperative virtual machine memory scheduling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,543,790 B2
- Document ID: US8543790B2
- Status: Granted
- Kind: B2
- Application number: 13/743,853
- Filed: 2013-01-17
- Published: 2013-06-06
- Granted: 2013-09-24
- Earliest claimed: 2006-04-27
- Assignees: VMware, Inc.
- Applicants: VMware, Inc.
- Inventors: Xiaoxin Chen; Carl A. Waldspruger; Anil Rao
- Web page: https://patentgrep.com/patent/US8543790B2

## CPC

- G06F 9/45545: {Guest-host, i.e. hypervisor is an application program itself, e.g. VirtualBox}
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 9/5016: {the resource being the memory}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 12/023: {Free address space management}
- G06F 12/08: in hierarchically structured memory systems, e.g. virtual memory systems
- G06F 12/1036: for multiple virtual address spaces, e.g. segmentation (G06F12/1045 takes precedence)
- G06F 12/109: for multiple virtual address spaces, e.g. segmentation (G06F12/1036 takes precedence)
- G06F 2009/45583: {Memory management, e.g. access or allocation}
- G06F 2209/508: Monitor
- G06F 2212/151: Emulated environment, e.g. virtual machine
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A resource scheduler for managing a distribution of host physical memory (HPM) among a plurality of virtual machines (VMs) monitors usage by each of the VMs of respective guest physical memories (GPM) to determine how much of the HPM should be allocated to each of the VMs. On determining that an amount of HPM allocated to a source VM should be reallocated to a target VM, the scheduler sends allocation parameters to a balloon application executing in the source VM causing it to reserve and write a value to a guest virtual memory (GVM) location in the source VM. The scheduler identifies the HPM location that corresponds to the reserved GVM and allocates it to the target VM by mapping a guest physical memory location of the target VM to the HPM location.

## Claims

### Claim 1

A method for implementation by a resource scheduler component of a hypervisor, the resource scheduler managing a distribution of host physical memory (HPM) among a plurality of virtual machines (VMs) executing by way of a hypervisor on a host computer system, each VM having guest software, the method comprising:
monitoring usage by each of the VMs of respective guest physical memories to determine how much of the HPM should be allocated to each of the VMs;
on determining that, based on said usage, an amount of HPM allocated to a source VM of the plurality of VMs should be reallocated to a target VM of the plurality of VMs, causing the resource scheduler to:
send allocation parameters to a balloon application executing in the source VM causing the balloon application executing in the source VM to reserve and write a value to a guest virtual memory (GVM) location in the source VM, the GVM location being mapped to a guest physical memory (GPM) location, which is mapped to an HPM location;
identifying the HPM location; and
allocating the HPM location to the target VM by mapping a guest physical memory location of the target VM to the HPM location.

### Claim 2 (depends on 1)

The method of claim 1, wherein the identifying of the HPM location comprises receiving the GPM location by way of a message from the balloon application, and converting the GPM location to the HPM location.

### Claim 3 (depends on 1)

The method of claim 1, wherein the identifying of the HPM location comprises:
searching HPM locations for the value;
adding HPM locations containing the value to a pool of HPM locations available for allocating to the plurality of VMs; and
selecting the HPM location from the pool of HPM locations for allocation to the target VM.

### Claim 4 (depends on 1)

The method of claim 1, wherein the allocating of the HPM location to the target VM comprises assigning the HPM location to a pool of available HPM locations for assigning to ones of the VMs that, according to the monitoring of the VMs, should have more HPM allocated.

### Claim 5 (depends on 1)

The method of claim 1, wherein the amount of HPM that should be reallocated to the target VM is based at least in part on relative priorities of VMs.

### Claim 6 (depends on 1)

The method of claim 1, wherein the amount of HPM that should be reallocated to the target VM is based at least in part on how much of the respective guest virtual memories of the source VM and the target VM are being actively used by the guest software executing within the source and target VMs.

### Claim 7 (depends on 6)

The method of claim 6, wherein the resource scheduler determines that more HPM should be allocated to the target VM when the target VM is actively using a greater percentage of its guest virtual memory than the source VM is using of its guest virtual memory.

### Claim 8 (depends on 1)

The method of claim 1, wherein a size of each memory location is one page of memory, wherein the writing of the value to the GVM location includes filling a page of memory corresponding to the GVM location with a pattern.

### Claim 9 (depends on 8)

The method of claim 8, wherein the writing of the value further includes filling the page of memory with all zeros.

### Claim 10 (depends on 1)

The method of claim 1, wherein the resource scheduler, on determining that the amount of HPM allocated to the source VM should be reallocated to the target VM, additionally marks the GPM location as read only in a pagetable for the source VM so that when the source VM attempts a write to the GPM location, a new HPM location is identified and a mapping is created from the GPM location to the new HPM location.

### Claim 11 (depends on 1)

The method of claim 1, wherein the resource scheduler, on determining that the amount of HPM allocated to the source VM should be reallocated to the target VM additionally unmaps the guest physical memory location by setting a page table entry for the GPM location to invalid.

### Claim 12

A host computer system having a memory system and a processor for executing instructions stored in the memory system, the instructions including a hypervisor that supports execution of a plurality of virtual machines (VMs) each having guest software, they hypervisor including a resource scheduler component for managing a distribution of host physical memory (HPM) among the VMs, the resource scheduler being configured to implement a method including steps of:
monitoring usage by each of the VMs of respective guest physical memories to determine how much of the HPM should be allocated to each of the VMs;
when an amount of HPM allocated to a source VM of the plurality of VMs should be reallocated to a target VM of the plurality of VMs:
sending allocation parameters to a balloon application executing in the source VM causing the balloon application executing in the source VM to reserve and write a value to a guest virtual memory (GVM) location in the source VM, the GVM location being mapped to a guest physical memory (GPM) location, which is mapped to an HPM location;
identifying the HPM location; and
allocating the HPM location to the target VM by mapping a guest physical memory location of the target VM to the HPM location.

### Claim 13 (depends on 12)

The host computer system of claim 12, wherein the identifying of the HPM location comprises receiving the GPM location by way of a message from the balloon application, and converting the GPM location to the HPM location.

### Claim 14 (depends on 12)

The host computer system of claim 12, wherein the identifying of the HPM location comprises:
searching HPM locations for the value;
adding HPM locations containing the value to a pool of HPM locations available for allocating to the plurality of VMs; and
selecting the HPM location from the pool of HPM locations for allocation to the target VM.

### Claim 15 (depends on 12)

The host computer system of claim 12, wherein the allocating of the HPM location to the target VM comprises assigning the HPM location to a pool of available HPM locations for assigning to ones of the VMs that, according to the monitoring of the VMs, should have more HPM allocated.

### Claim 16 (depends on 12)

The host computer system of claim 12, wherein the amount of HPM that should be reallocated to the target VM is based at least in part on relative priorities of VMs.

### Claim 17 (depends on 12)

The host computer system of claim 12, wherein the amount of HPM that should be reallocated to the target VM is based at least in part on how much of the respective guest virtual memories of the source VM and the target VM are being actively used by the guest software executing within the source and target VMs.

### Claim 18 (depends on 17)

The host computer system of claim 17, wherein the resource scheduler determines that more HPM should be allocated to the target VM when the target VM is actively using a greater percentage of its guest virtual memory than the source VM is using of its guest virtual memory.

### Claim 19 (depends on 12)

The host computer system of claim 12, wherein a size of each memory location is one page of memory, wherein the writing of the value to the GVM location includes filling a page of memory corresponding to the GVM location with a pattern.

### Claim 20 (depends on 19)

The host computer system of claim 19, wherein the writing of the value further includes filling the page of memory with all zeros.

### Claim 21 (depends on 12)

The host computer system of claim 12, wherein the resource scheduler, on determining that the amount of HPM allocated to the source VM should be reallocated to the target VM, additionally marks the GPM location as read only in a pagetable for the source VM so that when the source VM attempts a write to the GPM location, a new HPM location is identified and a mapping is created from the GPM location to the new HPM location.

### Claim 22 (depends on 12)

The host computer system of claim 12, wherein the resource scheduler, on determining that the amount of HPM allocated to the source VM should be reallocated to the target VM additionally unmaps the guest physical memory location by setting a page table entry for the GPM location to invalid.