# US 10,503,537 B2 · Hypervisor direct memory access

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,537 B2
- Document ID: US10503537B2
- Status: Granted
- Kind: B2
- Application number: 15/875,903
- Filed: 2018-01-19
- Published: 2019-03-21
- Granted: 2019-12-10
- Earliest claimed: 2017-09-19
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Aditya Bhandari; Bruce J. Sherwin, Jr.; Xin David Zhang
- Web page: https://patentgrep.com/patent/US10503537B2

## CPC

- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 12/1009: using page tables, e.g. page table structures
- G06F 12/1027: using associative or pseudo-associative address translation means, e.g. translation look-aside buffer [TLB]
- G06F 12/1036: for multiple virtual address spaces, e.g. segmentation (G06F12/1045 takes precedence)
- G06F 12/1081: for peripheral access to main memory, e.g. direct memory access [DMA]
- G06F 13/10: Program control for peripheral devices (G06F13/14 - G06F13/42 take precedence)
- G06F 13/28: using burst mode transfer, e.g. direct memory access {DMA}, cycle steal (G06F13/32 takes precedence)
- G06F 13/42: Bus transfer protocol, e.g. handshake; Synchronisation
- G06F 21/53: by executing in a restricted environment, e.g. sandbox or secure virtual machine
- G06F 2009/45566: {Nested virtual machines}
- G06F 2009/45579: {I/O management, e.g. providing access to device drivers or storage}
- G06F 2009/45583: {Memory management, e.g. access or allocation}
- G06F 2009/45587: {Isolation or security of virtual machine instances}
- G06F 2212/152: Virtualized environment, e.g. logically partitioned system
- G06F 2212/206: Memory mapped I/O
- G06F 2212/303: In peripheral interface, e.g. I/O adapter or channel
- G06F 2212/65: Details of virtual memory and virtual address translation
- G06F 2212/651: Multi-level translation tables
- G06F 2212/68: Details of translation look-aside buffer [TLB]
- G06F 2212/682: Multiprocessor TLB consistency
- G06F 2212/683: Invalidation
- G06F 2213/0026: PCI express
- G06F 2221/2149: Restricted operating environment
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

This disclosure generally relates to hypervisor memory virtualization. Techniques disclosed herein improve peripheral component interconnect express (PCI-e) device interoperability with a virtual machine. As an example, when a direct-memory access request is received from a PCI-e device but the target memory is currently unmapped, an indication may be provided to a memory paging processor so as to page-in the memory, such that the PCI-e device may continue to function normally. In some examples, the access request may be buffered and replayed once the memory is paged-in, or the access request may be retried, among other examples.

## Claims

### Claim 1

A system comprising:
at least one processor; and
a first memory storing instructions that, when executed by the at least one processor, perform a method for providing direct memory access, comprising:
identifying a direct memory access request for one or more pages of a second memory by a peripheral device;
communicating with a memory paging processor to determine whether the requested one or more pages are unmapped in the second memory;
based on determining that the requested one or more pages are unmapped in the second memory:
storing at least a part of the direct memory access request in a third memory as a buffered the direct memory access request; and
generating a request to the memory paging processor to page-in the one or more pages;
receiving, from the memory paging processor, an indication that the one or more pages are available in the second memory; and
in response to the indication that the one or more pages are available in the second memory, using the buffered direct memory access request, without requiring a retry direct memory access request from the peripheral device, to resume direct memory access by the peripheral device.

### Claim 2 (depends on 1)

The system of claim 1, wherein the direct memory access request is identified based on an indication received from an input-output memory management unit.

### Claim 3 (depends on 1)

The system of claim 1, wherein, based on resuming the direct memory access request, a retry direct memory access request for the one or more pages of memory is not received from the peripheral device.

### Claim 4 (depends on 1)

The system of claim 1, wherein the second memory and the third memory are subparts of a fourth memory.

### Claim 5 (depends on 1)

The system of claim 1, wherein the peripheral device is a peripheral component interconnect express device.

### Claim 6 (depends on 1)

The system of claim 1, wherein the request to the memory paging processor comprises at least one memory address associated with the one or more pages.

### Claim 7 (depends on 1)

The system of claim 1, wherein the direct memory access request is identified as a result of a virtual machine interacting with the peripheral device.

### Claim 8

A method for providing direct memory access for a device in use by a virtual machine, comprising:
identifying, as a result of a transaction between the device and the virtual machine, a memory access request for one or more pages of a first memory by the device;
communicating with a memory paging processor to determine whether the requested one or more pages are unmapped in the first memory;
based on determining that the requested one or more pages are unmapped in the first memory:
buffering at least a part of the memory access request in a second memory as a buffered memory access request; and
generating a request to the memory paging processor to page-in the one or more pages;
receiving, from the memory paging processor, an indication that the one or more pages are available in the first memory;
in response to the indication that the one or more pages are available in the first memory, continuing the transaction based on the buffered memory access request without requiring a retry memory access request from the device.

### Claim 9 (depends on 8)

The method of claim 8, wherein the memory access request is identified based on an indication received from an input-output memory management unit.

### Claim 10 (depends on 8)

The method of claim 8, wherein, based on continuing the transaction, a retry memory access request for the one or more pages of memory is not received from the device.

### Claim 11 (depends on 8)

The method of claim 8, wherein the first memory and the second memory are subparts of a third memory.

### Claim 12 (depends on 8)

The method of claim 8, wherein the request to the memory paging processor comprises at least one memory address associated with the one or more pages.

### Claim 13 (depends on 8)

The method of claim 8, wherein the device is a peripheral component interconnect express device.

### Claim 14

A method for providing direct memory access, comprising:
identifying a memory access request for one or more pages of a first memory by a peripheral device;
communicating with a memory paging processor to determine whether the requested one or more pages are unmapped in the first memory;
based on determining that the requested one or more pages are unmapped in the first memory:
storing at least a part of the memory access request in a second memory as a buffered memory access request; and
generating a request to the memory paging processor to page-in the one or more pages;
receiving, from the memory paging processor, an indication that the one or more pages are available in the first memory; and
in response to the indication that the one or more pages are available in the first memory, using the buffered memory access request, without requiring a retry memory access request from the peripheral device, to resume the memory access request for the one or more pages of the first memory.

### Claim 15 (depends on 14)

The method of claim 14, wherein the memory access request is identified based on an indication received from an input-output memory management unit.

### Claim 16 (depends on 14)

The method of claim 14, wherein the first memory and the second memory are subparts of a third memory.

### Claim 17 (depends on 14)

The method of claim 14, wherein the peripheral device is a peripheral component interconnect express device.

### Claim 18 (depends on 15)

The method of claim 15, wherein the request to the memory paging processor comprises at least one memory address associated with the one or more pages.

### Claim 19 (depends on 18)

The method of claim 18, wherein the at least one memory address is a physical memory address or a virtual memory address.

### Claim 20 (depends on 14)

The method of claim 14, wherein the memory access request is identified as a result of a virtual machine interacting with the peripheral device.