# US 2025/0097293 A1 · Nonstop virtual remote direct memory access

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0097293 A1
- Document ID: US20250097293A1
- Status: Published application
- Kind: A1
- Application number: 18/468,452
- Filed: 2023-09-15
- Published: 2025-03-20
- Earliest claimed: 2023-09-15
- Assignees: Hewlett Packard Enterprise Development LP
- Applicants: Hewlett Packard Enterprise Development LP
- Inventors: James R. Hamrick, JR.
- Web page: https://patentgrep.com/patent/US20250097293A1

## CPC

- H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
- G06F 9/45558: {Hypervisor-specific management and integration aspects}
- G06F 2009/45583: {Memory management, e.g. access or allocation}
- G06F 2009/45595: {Network integration; Enabling network access in virtual machine instances}
- H04L 69/161: {Implementation details of TCP/IP or UDP/IP stack architecture; Specification of modified or new header fields}
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method can facilitate software-based remote direct memory access (RDMA) communications over transport control protocol (TCP). The system establishes, by an RDMA emulation software layer in a first virtual machine, a TCP connection with a second virtual machine. Responsive to establishing the TCP connection, the system identifies, by the RDMA emulation software layer in the first virtual machine based on socket information received from the second virtual machine, a TCP socket associated with the TCP connection. Responsive to the first virtual machine initiating an RDMA send operation, the system, by the RDMA emulation software layer in the first virtual machine: prepends an RDMA send frame with a protocol header associated with the RDMA emulation software layer; and transmits, via the TCP socket, the prepended RDMA send frame. The system performs the RDMA send operation over the TCP connection.

## Claims

### Claim 1

A computer-implemented method, comprising:
establishing, by a remote direct memory access (RDMA) emulation software layer in a first virtual machine, a transport control protocol (TCP) connection with a second virtual machine;
responsive to establishing the TCP connection, identifying, by the RDMA emulation software layer in the first virtual machine based on socket information received from the second virtual machine, a TCP socket associated with the TCP connection;
responsive to the first virtual machine initiating an RDMA send operation:
prepending, by the RDMA emulation software layer in the first virtual machine, an RDMA send frame with a first protocol header associated with the RDMA emulation software layer; and
transmitting, by the RDMA emulation software layer in the first virtual machine via the TCP socket, the prepended RDMA send frame; and
performing the RDMA send operation over the TCP connection.

### Claim 2 (depends on 1)

The method of claim 1, the RDMA emulation software layer comprising:
a first software layer in the first virtual machine; and
a second software layer in the second virtual machine.

### Claim 3 (depends on 2)

The method of claim 2,
the first protocol header indicating a size of the RDMA send frame, wherein the method further comprises:
transmitting the RDMA send frame to the second virtual machine, which causes data associated with the RDMA send frame to be transferred into a receive message buffer associated with the second virtual machine;
receiving, by the first virtual machine from the second virtual machine, a cumulative acknowledgment of the RDMA send frame and any additional RDMA send frames associated with the RDMA send operation,
the cumulative acknowledgment prepended with a second protocol header associated with the RDMA emulation software layer, and the second protocol header indicating a size of the cumulative acknowledgment; and
notifying the first virtual machine that the RDMA send operation has completed.

### Claim 4 (depends on 3)

The method of claim 3,
the cumulative acknowledgment further prepended with a second RDMA send frame and additional data,
the second protocol header indicating a size of the second RDMA send frame, the additional data, and the cumulative acknowledgment, and
wherein the method further comprises:
transferring, by the first virtual machine, data associated with the second RDMA send frame into a receive message buffer associated with the first virtual machine; and
notifying the first virtual machine that receiving the data in the second RDMA send frame has completed.

### Claim 5 (depends on 2)

The method of claim 2, further comprising:
responsive to the first virtual machine initiating an RDMA Write operation:
prepending, by the first software layer, an RDMA Write frame with a second protocol header associated with the RDMA emulation software layer; and
transmitting, by the first software layer via the TCP socket, the prepended RDMA Write frame, which causes data associated with the RDMA Write frame to be transferred into an RDMA Write target buffer associated with the second virtual machine;
receiving, by the first virtual machine from the second virtual machine, a cumulative acknowledgment of the RDMA Write frame and any additional RDMA Write frames associated with the RDMA Write operation,
the cumulative acknowledgment prepended with a third protocol header associated with the RDMA emulation software layer, and the third protocol header indicating a size of the cumulative acknowledgment; and
notifying the first virtual machine that the RDMA Write operation has completed.

### Claim 6 (depends on 5)

The method of claim 5, further comprising one or more of:
the RDMA Write frame appended with additional data, and the prepended second protocol header indicating a size of the RDMA Write frame and the additional data; and
the cumulative acknowledgment further prepended with a second RDMA Write frame and additional data, the third protocol header indicating a size of the second RDMA Write frame, the additional data, and the cumulative acknowledgment, and
wherein the method further comprises:
transferring, by the first virtual machine, data associated with the second RDMA Write frame into an RDMA Write target buffer associated with the first virtual machine; and
notifying the first virtual machine that the RDMA Write operation has completed.

### Claim 7 (depends on 2)

The method of claim 2, further comprising:
responsive to the first virtual machine initiating an RDMA Read operation:
prepending, by the first software layer, an RDMA Read request with a second protocol header associated with the RDMA emulation software layer;
transmitting, by the first software layer via the TCP socket, the prepended RDMA Read request;
in response to the transmitting, the second virtual machine to obtain and return, in an RDMA Read reply frame, data requested in the RDMA Read request, the RDMA Read reply frame prepended with a third protocol header associated with the RDMA emulation software layer, and the third protocol header indicating a size of the RDMA Read reply frame;
receiving, by the first virtual machine, the prepended RDMA Read reply frame;
transferring, by the first virtual machine, the obtained data in the RDMA Read reply frame into a local buffer associated with the first virtual machine; and
notifying the first virtual machine that the RDMA Read operation has completed.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
the RDMA Read reply frame appended with additional data, and
the third protocol header indicating a size of the RDMA Read reply frame and the additional data.

### Claim 9 (depends on 2)

The method of claim 2, further comprising:
establishing the TCP connection responsive to the first virtual machine transmitting a first message to establish a communication channel with the second virtual machine;
transmitting, by the first virtual machine, a request to close the communication channel with the second virtual machine;
responsive to the transmitted request to close the communication channel, transmitting, by the first software layer to the second software layer, a first reset message which closes the TCP connection for the first virtual machine,
the transmitted request causing the second virtual machine to issue a response to the request and further causing the second software layer to transmit to the first software layer a second reset message which closes the TCP connection for the second virtual machine;
receiving, by the first virtual machine, the response to the request; and
notifying the first and the second virtual machines that the communication channel is closed.

### Claim 10 (depends on 9)

The method of claim 9, wherein identifying the TCP socket for transmitting data associated with the TCP connection comprises:
receiving, by the first software layer from the second software layer, an identifier for a queue pair corresponding to the TCP socket;
associating, by the first software layer, the TCP socket with the identifier for the queue pair;
transmitting, by the first software layer to the second software layer, an acknowledgment of the identifier; and
subsequent to exchanging a reply acknowledging the first message and an acknowledgment of the reply, notifying the first and the second virtual machines of a successfully established communication channel.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
performing one or more of the RDMA send operation, an RDMA Write operation, and an RDMA Read operation over the TCP connection without requiring use of a dedicated RDMA-enabled network interface card.

### Claim 12

A computer system, comprising:
a first virtual machine, comprising:
a first operating system associated with a first software layer which emulates a remote direct memory access (RDMA) protocol; and
a second operating system which implements transport control protocol (TCP) and communicates via a first hypervisor with a first Ethernet NIC associated with the second operating system; and
a second virtual machine, comprising:
a second software layer which emulates the RDMA protocol,
the second virtual machine implementing TCP and communicating via a second hypervisor with a second Ethernet NIC associated with the second virtual machine.

### Claim 13 (depends on 12)

The computer system of claim 12,
the first operating system and the second operating system communicating via a shared memory interface and inter-processor interrupts.

### Claim 14 (depends on 12)

The computer system of claim 12,
the first virtual machine and the second virtual machine communicating using the first Ethernet NIC and the second Ethernet NIC via one or more fabric switching networks.

### Claim 15 (depends on 12)

The computer system of claim 12,
wherein a multi-cloud orchestration tool deploys and manages the first virtual machine and the second virtual machine.

### Claim 16 (depends on 12)

The computer system of claim 12,
the first virtual machine comprising a virtual machine in a massively parallel processing (MPP) architecture,
the first operating system comprising an operating system in the MPP architecture,
the second virtual machine comprising a cluster input/output module virtual machine in the MPP architecture, and
the first software layer and the second software layer comprising an RDMA emulation software layer.

### Claim 17 (depends on 12)

The computer system of claim 12,
the first software layer and the second software layer deployed in the first and the second virtual machines based on at least one of:
a single root input/output virtualization;
a peripheral component interconnect pass-through; and
a fully virtualized system fabric interface.

### Claim 18

A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method, the method comprising:
establishing, by a remote direct memory access (RDMA) emulation software layer in a first virtual machine, a transport control protocol (TCP) connection with a second virtual machine;
responsive to establishing the TCP connection, identifying, by the RDMA emulation software layer in the first virtual machine based on socket information received from the second virtual machine, a TCP socket associated with the connection; and
responsive to the first virtual machine initiating an RDMA Write operation:
prepending, by the RDMA emulation software layer in the first virtual machine, an RDMA Write frame with a protocol header associated with the RDMA emulation software layer; and
transmitting, by the RDMA emulation software layer in the first virtual machine via the TCP socket, the prepended RDMA Write frame, which causes the data associated with the RDMA Write frame to be transferred into an RDMA Write target buffer associated with the second virtual machine; and
receiving, by the first virtual machine from the second virtual machine, a cumulative acknowledgment of the RDMA Write frame and any additional RDMA Write frames associated with the RDMA Write operation,
the cumulative acknowledgment prepended with a protocol header associated with the RDMA emulation software layer, and the protocol header indicating a size of the cumulative acknowledgment.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18,
the RDMA Write frame appended with additional data, and the prepended protocol header indicating a size of the RDMA Write frame and the additional data; and
the cumulative acknowledgment further prepended with a second RDMA Write frame and additional data,
the protocol header indicating a size of the second RDMA Write frame, the additional data, and the cumulative acknowledgment, and
wherein the method further comprises:
transferring, by the first virtual machine, data associated with the second RDMA Write frame into an RDMA Write target buffer associated with the first virtual machine; and
notifying the first virtual machine that the RDMA Write operation has completed.

### Claim 20 (depends on 18)

The non-transitory computer-readable storage medium of claim 18,
the RDMA emulation software layer comprising a first software layer in the first virtual machine and the RDMA emulation software layer further comprising a second software layer in the second virtual machine,
wherein identifying the TCP socket associated with the TCP connection comprises:
receiving, by the RDMA emulation software layer in the first virtual machine from RDMA the second software layer, an identifier for a queue pair corresponding to the TCP socket; and
associating, by the first software layer, the TCP socket with the identifier for the queue pair;
transmitting, by the first software layer to the second software layer, an acknowledgment of the identifier; and
subsequent to exchanging a reply acknowledging the first message and an acknowledgment of the reply, notifying the first and the second virtual machines of a successfully established communication channel.