# US 12,223,363 B2 · Performing workload migration in a virtualized system based on predicted resource distribution

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,363 B2
- Document ID: US12223363B2
- Status: Granted
- Kind: B2
- Application number: 17/547,944
- Filed: 2021-12-10
- Published: 2022-12-08
- Granted: 2025-02-11
- Earliest claimed: 2021-06-02
- Assignees: EMC IP Holding Company LLC
- Applicants: EMC IP Holding Company LLC
- Inventors: Jie Huang; Guoping Guan; Yunyun Duan
- Web page: https://patentgrep.com/patent/US12223363B2

## CPC

- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 9/505: {considering the load}
- G06F 9/45533: {Hypervisors; Virtual machine monitors}
- G06F 9/4856: {resumption being on a different machine, e.g. task migration, virtual machine migration (G06F9/5088 takes precedence)}
- G06F 9/4875: {with migration policy, e.g. auction, contract negotiation}
- G06F 9/5088: {involving task migration}
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/08: Learning methods
- G06F 2009/45591: {Monitoring or debugging support}
- G06F 2209/5019: Workload prediction
- G06F 2209/5022: Workload threshold
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Techniques for managing resources of a virtualized system involve acquiring historical distribution data about a virtualized system, the historical distribution data indicating a historical distribution of resources occupied by workloads on a plurality of host machines of the virtualized system over a predetermined historical time period. The techniques further involve generating predicted distribution data based on the historical distribution data, the predicted distribution data indicating an estimated distribution of resources occupied by the workloads on the plurality of host machines over a predetermined future time period. The techniques further involve performing workload migration at least once based on the predicted distribution data, the workload migration including migrating a workload of a first host machine whose first estimated quantity of occupied resources exceeds a high threshold to a second host machine whose second estimated quantity of occupied resources is below a low threshold.

## Claims

### Claim 1

A method, comprising:
acquiring historical distribution data about a virtualized system, the historical distribution data indicating a historical distribution of resources occupied by virtual machines on each of a plurality of host machines of the virtualized system over a predetermined historical time period;
generating predicted distribution data based on the historical distribution data, the predicted distribution data indicating an estimated distribution of resources occupied by the virtual machines on each of the plurality of host machines over a predetermined future time period, wherein a first host machine of the plurality of host machines has a first estimated quantity of occupied resources that exceeds a high threshold and a second host machine of the plurality of host machines has a second estimated quantity of occupied resources that is below a low threshold; and
when at least one virtual machine of the first host machine occupies a quantity of resources not higher than a difference between the first estimated quantity of occupied resources and the second estimated quantity of occupied resources:
migrating, at least once based on the predicted distribution data, a most appropriate virtual machine of the first host machine to the second host machine, wherein the most appropriate virtual machine comprises a virtual machine of the first host machine which results in a residual quantity of occupied resources of the second host machine after the migration that is closest to
an average of the first estimated quantity of occupied resources and the second estimated quantity of occupied resources as compared to migration of other individual virtual machines of the first host machine.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
stopping the migrating from being initially performed when the quantity of resources occupied by each virtual machine of the first host machine occupies a quantity of resources that is higher than the difference between the first estimated quantity of occupied resources and the second estimated quantity of occupied resources.

### Claim 3 (depends on 2)

The method according to claim 2, further comprising, after the stopping of the migrating from being initially performed, subsequently migrating the most appropriate virtual machine of the first host machine to the second host machine when the quantity of resources occupied by at least one virtual machine of on the first host machine is not higher than the difference between the first estimated quantity of occupied resources and the second estimated quantity of occupied resources.

### Claim 4 (depends on 1)

The method according to claim 1, wherein generating the predicted distribution data based on the historical distribution data includes:
generating the predicted distribution data using the historical distribution data as a long short-term memory (LSTM) machine learning model.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the most appropriate workload of the first host machine results in a residual quantity of occupied resources of the second host machine after the migration that is closest to the average of the first estimated quantity of occupied resources and the second estimated quantity of occupied resources based on i) the second estimated quantity of occupied resources and ii) for each virtual machine of the first machine, a quantity of resources occupied by that virtual machine.

### Claim 6 (depends on 1)

The method according to claim 1, further comprising:
receiving resource requirements of the virtual machines submitted by a user to a virtual resource manager.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the virtual resource manager includes a processor, an input/output interface, a memory, and a network interface.

### Claim 8 (depends on 7)

The method according to claim 7, wherein a resource schedule associated with the virtual resource manager is based on resources of the plurality of host machines including the first host machine and the second host machine that are within a virtual resource pool, the resources of the plurality of host machines including computing resources, communication resources, and storage resources.

### Claim 9

An electronic device, comprising:
a processor; and
a memory coupled to the processor, the memory having instructions stored therein that, when executed by the processor, cause the device to execute actions including:
acquiring historical distribution data about a virtualized system, the historical distribution data indicating a historical distribution of resources occupied by virtual machines on each of a plurality of host machines of the virtualized system over a predetermined historical time period;
generating predicted distribution data based on the historical distribution data, the predicted distribution data indicating an estimated distribution of resources occupied by the virtual machines on each of the plurality of host machines over a predetermined future time period, wherein a first host machine of the plurality of host machines has a first estimated quantity of occupied resources that exceeds a high threshold and a second host machine of the plurality of host machines has a second estimated quantity of occupied resources that is below a low threshold; and
when at least one virtual machine of the first host machine occupies a quantity of resources not higher than a difference between the first estimated quantity of occupied resources and the second estimated quantity of occupied resources:
migrating, at least once based on the predicted distribution data, a most appropriate virtual machine of the first host machine to the second host machine, wherein the most appropriate virtual machine comprises a virtual machine of the first host machine which results in a residual quantity of occupied resources of the second host machine after the migration that is closest to
an average of the first estimated quantity of occupied resources and the second estimated quantity of occupied resources as compared to migration of other individual virtual machines of the first host machine.

### Claim 10 (depends on 9)

The electronic device according to claim 9, wherein the electronic device is caused to execute actions further including:
stopping the migrating from being initially performed when the quantity of resources occupied by each virtual machine of the first host machine occupies a quantity of resources that is higher than the difference between the first estimated quantity of occupied resources and the second estimated quantity of occupied resources.

### Claim 11 (depends on 9)

The electronic device according to claim 9, wherein generating the predicted distribution data based on the historical distribution data includes:
generating the predicted distribution data using the historical distribution data as a long short-term memory (LSTM) machine learning model.

### Claim 12

A computer program product having a non-transitory computer readable medium which stores a set of instructions, the set of instructions, when carried out by computerized circuitry, causing the computerized circuitry to perform a method of:
acquiring historical distribution data about a virtualized system, the historical distribution data indicating a historical distribution of resources occupied by virtual machines on each of a plurality of host machines of the virtualized system over a predetermined historical time period;
generating predicted distribution data based on the historical distribution data, the predicted distribution data indicating an estimated distribution of resources occupied by the virtual machines on each of the plurality of host machines over a predetermined future time period, wherein a first host machine of the plurality of host machines has a first estimated quantity of occupied resources that exceeds a high threshold and a second host machine of the plurality of host machines has a second estimated quantity of occupied resources that is below a low threshold; and
when at least one virtual machine of the first host machine occupies a quantity of resources not higher than a difference between the first estimated quantity of occupied resources and the second estimated quantity of occupied resources:
migrating, at least once based on the predicted distribution data, a most appropriate virtual machine of the first host machine to the second host machine, wherein the most appropriate virtual machine comprises a virtual machine of the first host machine which results in a residual quantity of occupied resources of the second host machine after the migration that is closest to
an average of the first estimated quantity of occupied resources and the second estimated quantity of occupied resources as compared to migration of other individual virtual machines of the first host machine.