# US 2017/0364279 A1 · Systems and methods for non-uniform memory access aligned i/o for virtual machines

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0364279 A1
- Document ID: US20170364279A1
- Status: Published application
- Kind: A1
- Application number: 15/183,386
- Filed: 2016-06-15
- Published: 2017-12-21
- Earliest claimed: 2016-06-15
- Assignees: VMware, Inc.
- Applicants: VMware, Inc.
- Inventors: Amitabha Banerjee; Rishi Mehta; Xiaochuan Shen; Seongbeom Kim
- Web page: https://patentgrep.com/patent/US20170364279A1

## CPC

- G06F 3/0611: {in relation to response time}
- G06F 3/0659: {Command handling arrangements, e.g. command buffers, queues, command scheduling}
- G06F 3/0664: {at device level, e.g. emulation of a storage device or system}
- G06F 3/067: {Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]}
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues}
- G06F 9/5077: {Logical partitioning of resources; Management or configuration of virtualized resources (specific details on emulation or internal functioning of virtual machines G06F9/455)}
- G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage}
- G06F 2009/45583: {Memory management, e.g. access or allocation}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and methods described herein align various types of hypervisor threads with a non-uniform memory access (NUMA) client of a virtual machine (VM) that is driving I/O transactions from an application so that no remote memory access is required and the I/O transactions can be completed with local accesses to CPUs, caches, and the I/O devices of a same NUMA node of a hardware NUMA system. First, hypervisor of the VM detects whether the VM runs on a single or multiple NUMA nodes. If the VM runs on multiple NUMA nodes, a NUMA client on which the application is executing the I/O transactions is identified and knowledge of resource sharing between the NUMA client and its related hypervisor threads is established. Such knowledge is then utilized to schedule the NUMA client and its related hypervisor threads to the same NUMA node of the NUMA system.

## Claims

### Claim 1

An system to support non-uniform memory access (NUMA) aligned I/O, comprising:
a NUMA hardware system having a hypervisor running on it and configured to:
determine whether a virtual machine (VM) is assigned to a single or multiple NUMA nodes of the NUMA hardware system, wherein each of the NUMA nodes comprises a group of local memories and CPUs having the same memory access latency;
identify a NUMA client of the VM on which an application is executing I/O transactions if the VM runs on multiple NUMA nodes, wherein the VM is grouped into a plurality of NUMA clients each having no more virtual CPUs than the CPUs in each of the NUMA nodes;
determine resource sharing between the NUMA client and its related hypervisor threads;
utilize the determination of resource sharing to schedule the NUMA client and its related hypervisor threads to a same NUMA node of the NUMA system;
move one or more IO processing threads of a virtual I/O device to the NUMA client so that all of the I/O transactions, the NUMA client of the VM, the IO processing threads of the virtual NIC, and the related hypervisor threads NUMA client are aligned on the same NUMA node of the NUMA system.

### Claim 2 (depends on 1)

The system of claim 1, wherein:
the hypervisor is configured to optimize memory allocation for the I/O transactions of the VM running on the NUMA system by utilizing NUMA hardware configuration information from native OS of the NUMA system.

### Claim 3 (depends on 1)

The system of claim 1, wherein:
the hypervisor threads include one or more of:
VM transmitting threads configured to transmit packets of the I/O transactions between the application and a physical network interface card;
network polling thread configured to poll packets from an I/O device of the NUMA hardware system and supplying them to the VM;
network scheduling thread configured to arbitrate among the different VM transmitting threads, all of which try to transmit the packets to a same device queue of the I/O device.

### Claim 4 (depends on 3)

The system of claim 3, wherein:
the hypervisor is configured to establish an association between the VM transmitting thread and the VM if the VM is assigned to a single NUMA node, wherein such association is provided to the hypervisor as an indication to schedule the VM and the corresponding VM transmitting thread on the same physical NUMA node.

### Claim 5 (depends on 4)

The system of claim 4, wherein:
the hypervisor is configured to identify relationships between the VM and its corresponding network polling thread and network scheduler thread via signaling mechanism between the hypervisor and guest OS of the VM.

### Claim 6 (depends on 1)

The system of claim 1, wherein:
the hypervisor is configured to identify the NUMA client by inspecting a network transmit descriptor and/or receive descriptor of the virtual NIC, wherein the descriptor is configured to maintain header and/or payload of one or more packets of the I/O transactions.

### Claim 7 (depends on 1)

The system of claim 1, wherein:
the hypervisor is configured to set a communication tracking rate when one or more packets from the I/O transactions are transmitted and received or when the I/O transactions are done, wherein the communication tracking rate is an indication provided to the hypervisor that a communication link exists between the NUMA client of the VM and its related hypervisor threads.

### Claim 8

A method to support non-uniform memory access (NUMA) aligned I/O, comprising:
determining whether a virtual machine (VM) is assigned to a single or multiple NUMA nodes of the NUMA hardware system, wherein each of the NUMA nodes comprises a group of local memories and CPUs having the same memory access latency;
identifying a NUMA client of the VM on which an application is executing I/O transactions if the VM runs on multiple NUMA nodes, wherein the VM is grouped into a plurality of NUMA clients each having no more virtual CPUs than the CPUs in each of the NUMA nodes;
determining resource sharing between the NUMA client and its related hypervisor threads;
utilizing the determination of resource sharing to schedule the NUMA client and its related hypervisor threads to a same NUMA node of the NUMA system;
moving one or more IO processing threads of a virtual I/O device to the NUMA client so that all of the I/O transactions, the NUMA client of the VM, the IO processing threads of the virtual NIC, and the related hypervisor threads of the NUMA client are aligned on the same NUMA node of the NUMA system.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
optimizing memory allocation for the I/O transactions of the VM running on the NUMA system by utilizing NUMA hardware configuration information from native OS of the NUMA system.

### Claim 10 (depends on 8)

The method of claim 8, wherein:
the hypervisor threads include one or more of:
VM transmitting threads configured to transmit packets of the I/O transactions between the application and a physical network interface card;
network polling thread configured to poll packets from an I/O device of the NUMA hardware system and supplying them to the VM;
network scheduling thread configured to arbitrate among the different VM transmitting threads, all of which try to transmit the packets to a same device queue of the I/O device.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
establishing an association between the VM transmitting thread and the VM if the VM is assigned to a single NUMA node, wherein such association is provided to the hypervisor as an indication to schedule the VM and the corresponding VM transmitting thread on the same physical NUMA node.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
identifying relationships between the VM and its corresponding network polling thread and network scheduler thread via signaling mechanism between the hypervisor and guest OS of the VM.

### Claim 13 (depends on 8)

The method of claim 8, further comprising:
identifying the NUMA client by inspecting a network transmit descriptor and/or receive descriptor of the virtual NIC, wherein the descriptor is configured to maintain header and/or payload of one or more packets of the I/O transactions.

### Claim 14 (depends on 8)

The method of claim 8, further comprising:
setting a communication tracking rate when one or more packets from the I/O transactions are transmitted and received or when the I/O transactions are done, wherein the communication tracking rate is an indication provided to the hypervisor that a communication link exists between the NUMA client of the VM and its related hypervisor threads.

### Claim 15

At least one computer-readable storage medium having computer-executable instructions embodied thereon, wherein, when executed by at least one processor, the computer-executable instructions cause the at least one processor to:
determine whether a virtual machine (VM) is assigned to a single or multiple non-uniform memory access (NUMA) nodes of the NUMA hardware system, wherein each of the NUMA nodes comprises a group of local memories and CPUs having the same memory access latency;
identify a NUMA client of the VM on which an application is executing I/O transactions if the VM runs on multiple NUMA nodes, wherein the VM is grouped into a plurality of NUMA clients each having no more virtual CPUs than the CPUs in each of the NUMA nodes;
determine resource sharing between the NUMA client and its related hypervisor threads;
utilize the determination of resource sharing to schedule the NUMA client and its related hypervisor threads to a same NUMA node of the NUMA system;
move one or more IO processing threads of a virtual I/O device to the NUMA client so that all of the I/O transactions, the NUMA client of the VM, the IO processing threads of the virtual NIC, and the related hypervisor threads of the NUMA client are aligned on the same NUMA node of the NUMA system.

### Claim 16 (depends on 15)

The at least one computer-readable storage medium of claim 15, wherein the computer-executable instructions further cause the at least one processor to:
optimize memory allocation for the I/O transactions of the VM running on the NUMA system by utilizing NUMA hardware configuration information from native OS of the NUMA system.

### Claim 17 (depends on 15)

The at least one computer-readable storage medium of claim 15, wherein the computer-executable instructions further cause the at least one processor to:
establish an association between a VM transmitting thread of the hypervisor threads and the VM if the VM is assigned to a single NUMA node, wherein such association is provided to the hypervisor as an indication to schedule the VM and the corresponding VM transmitting thread on the same physical NUMA node.

### Claim 18 (depends on 16)

The at least one computer-readable storage medium of claim 16, wherein the computer-executable instructions further cause the at least one processor to:
identify relationships between the VM and its corresponding network polling thread and network scheduler thread via signaling mechanism between the hypervisor and guest OS of the VM.

### Claim 19 (depends on 15)

The at least one computer-readable storage medium of claim 15, wherein the computer-executable instructions further cause the at least one processor to:
identify the NUMA client by inspecting a network transmit descriptor and/or receive descriptor of the virtual NIC, wherein the descriptor is configured to maintain header and/or payload of one or more packets of the I/O transactions.

### Claim 20 (depends on 15)

The at least one computer-readable storage medium if claim 15, wherein the computer-executable instructions clause the at least one processor to:
set a communication tracking rate when one or more packets from the I/O transactions are transmitted and received or when the I/O transactions are done, wherein the communication tracking rate is an indication provided to the hypervisor that a communication link exists between the NUMA client of the VM and its related hypervisor threads.
