# US 10,503,679 B2 · NVM express controller for remote access of memory and I/O over Ethernet-type networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,679 B2
- Document ID: US10503679B2
- Status: Granted
- Kind: B2
- Application number: 15/242,299
- Filed: 2016-08-19
- Published: 2016-12-08
- Granted: 2019-12-10
- Earliest claimed: 2013-06-26
- Assignees: CNEX Labs, Inc.
- Applicants: CNEX Labs, Inc.
- Inventors: Yiren Ronnie Huang
- Web page: https://patentgrep.com/patent/US10503679B2

## CPC

- G06F 13/4068: {Electrical coupling}
- G06F 11/201: {between storage system components}
- G06F 11/3027: {where the computing system component is a bus}
- G06F 11/349: {for interfaces, buses}
- G06F 12/0835: {for main memory peripheral accesses (e.g. I/O or DMA)}
- G06F 13/00: Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units (interface circuits for specific input/output devices G06F3/00 {; multiprogram control therefor G06F9/46}; multiprocessor systems G06F15/16 )
- G06F 13/161: {with latency improvement}
- G06F 13/1684: {using multiple buses}
- G06F 13/42: Bus transfer protocol, e.g. handshake; Synchronisation
- G06F 13/4221: {being an input/output bus, e.g. ISA bus, EISA bus, PCI bus, SCSI bus}
- G06F 15/17337: {Direct connection machines, e.g. completely connected computers, point to point communication networks (coupling between buses G06F13/4004)}
- H04L 61/457: {containing identifiers of data entities on a computer, e.g. file names}
- H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
- G06F 16/10: File systems; File servers
- G06F 2213/0026: PCI express
- H04L 12/40176: {involving redundancy (error detection or correction of the data by redundancy in hardware using active fault-masking in interconnections G06F11/2002; error detection or correction of the data by redundancy in hardware using active fault-masking in storage systems using spares or by reconfiguring G06F11/2053)}
- H04L 2101/622: Layer-2 addresses, e.g. medium access control [MAC] addresses
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

A method and system for enabling Non-Volatile Memory express (NVMe) for accessing remote solid state drives (SSDs) (or other types of remote non-volatile memory) over the Ethernet or other networks. An extended NVMe controller is provided for enabling CPU to access remote non-volatile memory using NVMe protocol. The extended NVMe controller is implemented on one server for communication with other servers or non-volatile memory via Ethernet switch. The NVMe protocol is used over the Ethernet or similar networks by modifying it to provide a special NVM-over-Ethernet frame.

## Claims

### Claim 1

An extended NVMe controller comprising:
a host interface adapted to couple the extended NVMe controller to a host processor;
a direct network interface adapted to couple the extended NVMe controller to an external Ethernet network; and
wherein the extended NVMe controller is configured to:
receive, from the host processor, NVMe commands directed to a remote namespace for a remote non-volatile memory coupled to the external Ethernet network;
translate NVMe commands to NVMe over Ethernet (NVMoE) commands suitable for transmission over the external Ethernet network to a remote namespace controller for a remote namespace, the NVMe commands received from the host processer via the host interface, wherein the translation of the NVME command to NVMoE command comprises mapping a name space identifier (NSID) of the NVMe commands to an Ethernet media access control (MAC) address; and
transmit the translated NVMe command directly over the external Ethernet network to the remote namespace controller.

### Claim 2 (depends on 1)

The extended NVMe controller of claim 1, further comprising:
a mapping table that maps one or more NISDs to one or more Ethernet MAC addresses; and
wherein the extended NVMe controller maps the NISD of the NVMe commands to the Ethernet MAC address using the mapping table.

### Claim 3 (depends on 1)

The extended NVMe controller of claim 1 further comprising a scheduler and arbiter to schedule a queue for processing the NVMe commands.

### Claim 4 (depends on 1)

The extended NVMe controller of claim 1, further comprising a local memory interface adapted to couple the extended NVMe controller to a local namespace for local non-volatile memory, wherein the extended NVMe controller receives from the host processor NVMe commands directed to the local namespace and provides I/O access for the host processor to the local namespace.

### Claim 5 (depends on 4)

The extended NVMe controller of claim 4, wherein the NVMe commands address the local namespace by HSID and NSID, and the extended NVMe controller accesses a directory server that manages HSIDs and NSIDs to obtain HSIDs and/or NSIDs for the local namespace.

### Claim 6 (depends on 4)

The extended NVMe controller of claim 4, wherein the extended NVMe controller is coupled to a second extended NVMe controller, wherein the second extended NVMe controller is coupled to a second host processor, wherein the second extended NVMe controller is connected to the extended NVMe controller via a second direct network interface, and wherein the second extended NVMe controller is configured to mirror the local namespace of the extended NVMe controller and act as a standby if the extended NVMe controller fails.

### Claim 7 (depends on 5)

The extended NVMe controller of claim 5, wherein the extended NVMe controller requests one or more HSIDs and one or more NSIDs from the directory server, and receives HSIDs and NSIDs that are not repeated in the external Ethernet network.

### Claim 8 (depends on 1)

The extended NVMe controller of claim 1, further comprising a shared buffer pool to provide flow control on command and data flows from the direct network interface.

### Claim 9 (depends on 1)

The extended NVMe controller of claim 1, wherein the external Ethernet network includes a layer 3 network, and wherein the extended NVMe controller is further configured to encapsulate the NVMoE commands within layer 3 packet headers for transmission over the layer 3 network.

### Claim 10

A method for accessing a remote name space, the method comprising:
receiving, at an extended NVMe controller from a host processor, NVMe commands directed to a remote namespace for a remote non-volatile memory coupled to an external network, the extended NVMe controller also coupled to the external network, the extended NVMe controller coupled to the host processor via a host interface;
translating, at the extended NVMe controller, the received NVMe commands to NVMe over Ethernet (NVMoE) commands suitable for transmission over the external Ethernet network to a remote namespace controller for a remote namespace, wherein the translation of the NVME command to NVMoE command comprises mapping a name space identifier (NSID) of the NVMe commands to an Ethernet media access control (MAC) address; and
transmitting, at the extended NVMe controller, the translated NVMe command directly over the external Ethernet network to the remote namespace controller.

### Claim 11 (depends on 10)

The method of claim 10, wherein the extended NVMe controller further comprises:
a mapping table that maps one or more NISDs to one or more Ethernet MAC addresses; and
wherein the extended NVMe controller maps the NISD of the NVMe commands to the Ethernet MAC address using the mapping table.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
processing the received NVMe commands using a command scheduler and a command arbiter.

### Claim 13 (depends on 10)

The method of claim 10, further comprising:
receiving, by the extended NVMe controller from the host processor, NVMe commands directed to a local namespace;
providing, by the extended NVMe controller, I/O access for the host processor to the local namespace, wherein the extended NVMe controller comprises a local memory interface adapted to couple the extended NVMe controller to the local namespace for local non-volatile memory.

### Claim 14 (depends on 13)

The method of claim 13, wherein the NVMe commands address the local namespace by HSID and NSID, the method further comprising:
accessing a directory server that manages HSIDs and NSIDs to obtain HSIDs and/or NSIDs for the local namespace.

### Claim 15 (depends on 13)

The method of claim 13, wherein the extended NVMe controller is coupled to a second extended NVMe controller, wherein the second extended NVMe controller is coupled to a second host processor, wherein the second extended NVMe controller is connected to the extended NVMe controller via a second direct network interface, and wherein the second extended NVMe controller is configured to mirror the local namespace of the extended NVMe controller and act as a standby if the extended NVMe controller fails.

### Claim 16 (depends on 14)

The method of claim 14, further comprising:
requesting, by the extended NVMe controller, one or more HSIDs and one or more NSIDs from the directory server; and
receiving HSIDs and NSIDs that are not repeated in the external Ethernet network.

### Claim 17 (depends on 10)

The method of claim 10, further comprising:
providing flow control on command and data flows from the direct network interface using a shared buffer pool.

### Claim 18 (depends on 10)

The method of claim 10, wherein the external Ethernet network includes a layer 3 network, the method further comprising:
encapsulating, by the extended NVMe controller, the NVMoE commands within layer 3 packet headers for transmission over the layer 3 network.