US 7,627,693 B2 · IP storage processor and engine therefor using RDMA

Abstract

An IP Storage processor and processing engines for use in the IP storage processor is disclosed. The IP Storage processor uses an architecture that may provide capabilities to transport and process Internet Protocol (IP) packets from Layer 2 through transport protocol layer and may also perform packet inspection through Layer 7. The engines may perform pass-through packet classification, policy processing and/or security processing enabling packet streaming through the architecture at nearly the full line rate. A scheduler schedules packets to packet processors for processing. An internal memory or local session database cache may store a transport protocol session information database and/or store a storage information session database, for a certain number of active sessions. The session information that is not in the internal memory is stored and retrieved to/from an additional memory. An application running on an initiator or target can in certain instantiations register a region of memory, which is made available to its peer(s) for access directly without substantial host intervention through RDMA data transfer.

Claims55

Dependent claims

A hardware processor providing Transmission Control Protocol (TCP) or Stream Control Transmission Protocol (SCTP) or User Datagram Protocol (UDP) operations, or a combination of any of the foregoing, over an Internet Protocol (IP), said operations including remote direct memory access (RDMA) capability for data transfer over a network from or to an initiator and to or from a target, said operations requested by a host processor having a Small Computer Systems Interface (SCSI) command layer and an iSCSI driver or an IP Storage driver, said hardware processor comprising:

a. an RDMA mechanism;
b. a command scheduler for scheduling commands from the command layer of said host processor for operation in said hardware processor;
c. first command queues for queuing commands from said host processor for existing sessions;
d. second command queues for queuing commands from said host processor for sessions that do not currently exist;
e. a database for recording a state of the session on which said command is transported, said database also for recording progress of RDMA for those of said commands that use RDMA;
f. a communication path between said processor and said SCSI layer for communicating status of command execution to said SCSI layer for processing;
and
g. at least one transmit/receive engine and at least one command engine coupled to work together to interpret commands and perform appropriate operations for performing RDMA for storing/retrieving data to/from or transmitting/receiving data to/from said target or said initiator.

5 more elements

A system for a programmable Internet Protocol (IP) Storage processor engine, said processor having Remote Direct Memory Access (RDMA) capability, and used for processing IP Storage Protocol Data Units (PDUs) transported over an IP network using Internet Protocol packets having fields including a plurality of PDUs with fields, said system comprising:

a cyclic redundancy check (CRC) component for IP Storage PDU CRC verification or for new CRC generation or a combination of any of the foregoing;
a data memory for storing said packets and/or said protocol data units;
a packet look-up interface for providing access to said packets or PDUs or packet fields or PDU fields or a combination of any of the foregoing;
an execution resource;
an instruction decoder and an IP Storage PDU classifier to direct IP Storage processor engine operation;
a sequence manager providing specific sequence operations for use in IP Storage data sequencing calculations;
and further comprising:
an IP Storage session manager including a hash engine to perform hash operations against predetermined fields of the packet to perform a hash table walk to determine the correct session entry for said packet;
a register file for extracting predetermined header fields from said packets and PDUs for IP Storage processing;
pointer registers for indicating data source and destination;
and
context register sets for holding multiple contexts for packet execution;
said multiple contexts allowing, in response to a given packet execution stalling, another context to be invoked to enable said IP Storage processor engine to continue the execution of another packet stream;
said IP Storage processor engine having a cache for holding recently or frequently used session entries, including connection IDs, for local use and further having an interface for informing a packet scheduler of the connection identifiers that are cached for each IP Storage processor engine resource.

11 more elements

A hardware implemented Internet Protocol (IP) storage controller providing a remote direct memory access (RDMA) useable in one of a Transmission Control Protocol (TCP), a Stream Control Transmission Protocol (SCTP), and a User Datagram Protocol (UDP) over IP, said IP Storage controller for transporting a plurality of Internet Small Computer Systems Interface (iSCSI) commands and a plurality of Protocol Data Units (PDUs), said IP Storage controller having access to a database for keeping track of data processing operations, said database located in one of a memory on said IP Storage controller and a memory partly on said IP Storage controller and partly in a computing apparatus other than said IP Storage controller, said IP Storage controller coupled to a host processor having an SCSI command layer and an IP storage driver, said IP Storage controller having a transmit and a receive path for data flow, said IP Storage controller comprising:

a. a command scheduler configured to schedule processing of the iSCSI commands, said scheduler coupled to said SCSI command layer and to said IP storage driver;
b. a receive path configured to facilitate a data flow of received data and a transmit path configured to facilitate a data flow of transmitted data, wherein the received data and the transmitted data are related to the iSCSI commands;
c. at least one transmit engine configured to transmit the PDUs related to the transmitted data;
d. at least one transmit command engine configured to interpret said PDUs and configured to perform a plurality of operations comprising retrieving information from said host processor to service the iSCSI commands and configured to keep command flow data related to the iSCSI commands in said database updated as said retrieving progresses;
e. at least one receive command engine for receiving said iSCSI commands;
and
f. at least one receive command engine for interpreting said commands and configured to perform a plurality of operations comprising:
storing the information to the host processor to service one of the iSCSI commands;
and
keeping the command flow data in said database updated as the one of the storing and the retrieving progresses.

7 more elements

A system comprising:

a hardware processor configured to transfer data by executing at least one of a Transmission Control protocol (TCP), a Stream Control Transmission Protocol (SCTP), and a User Datagram Protocol (UDP) over an Internet Protocol (IP), said hardware processor configured to execute a remote direct memory access (RDMA) to perform the transfer of the data over a network between an initiator and target;
and
a host processor having a Small Computer Systems Interface (SCSI) layer and an IP Storage driver, said host processor configured to request the transfer of the data, said SCSI layer further comprising an SCSI command layer, said hardware processor comprising:
a. a command scheduler configured to schedule a plurality of commands received from the SCSI command layer of said host processor to operate in said hardware processor;
b. a first set of command queues configured to queue a first subset of the commands received from said host processor, the first subset of the commands relating to a plurality of existing sessions;
c. a second set of command queues configured to queue a second subset of the commands received from said host processor, the second subset of the commands relating to a plurality of nonexisting sessions that do not exist;
d. a database configured to record a state of one of the existing sessions on which one of the commands is transported, said database also further configured to record a progress of RDMA for one of the commands that use RDMA;
e. a communication path between said hardware processor and said SCSI layer, said communication path configured to facilitate a communication of status of execution of the commands to said SCSI layer;
and
f. at least one transmit/receive engine and at least one command engine coupled to each other, said at least one transmit/receive engine and at least one command engine configured to work together to interpret the commands and perform operations to execute RDMA to access the data from at least one of said target and said initiator.

7 more elements

A programmable Internet Small Computer System Interface (iSCSI) processor implemented on a chip having RDMA capability, said iSCSI processor chip for processing Internet Protocol (IP) packets during TCP/IP sessions in accordance with TCP/IP session information, said IP packets comprising at least one of:

a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields;
said iSCSI processor chip having a memory for storing or retrieving said Internet Protocol Packets or said TCP/IP session information, and said iSCSI processor chip capable of being coupled to a host system, said host system comprising one or more host processors and host buffers, said iSCSI processor chip comprising:
an iSCSI processing engine for offloading iSCSI processing from said one or more host processors to perform iSCSI data transfers;
a memory controller for controlling access to said memory for storing or retrieving said TCP/IP session information or said Internet Protocol packets;
an execution resource for generating or processing said TCP/IP header and field information and for performing TCP/IP operations on said Internet Protocol packets;
a network interface to send or receive said Internet Protocol packets from or to said iSCSI processor chip;
a host interface controller to couple said iSCSI processor chip to said one or more host processors;
an RDMA execution resource for generating or processing said RDMA header and field information and for performing RDMA operations on said Internet Protocol packets that require RDMA operations;
and
a transmit engine or a receive engine, or a combination thereof, coupled to one of said one or more host processors for retrieving data from said host buffers or performing direct data placement or zero copy in said host buffers without substantial intervention by said host processor.

7 more elements

An Internet Small Computer Systems Interface (iSCSI) processor implemented on a chip having a remote direct memory access (RDMA) capability, said iSCSI processor configured to process Internet Protocol (IP) packets comprising at least one of:

a) IP Storage Protocol Data Units, b) TCP/IP headers and fields and c) RDMA headers and fields, said iSCSI processor comprising:
a. an RDMA mechanism configured to perform RDMA on a set of data;
b. Cyclic Redundancy Check (CRC) hardware for performing a plurality of CRC functions on the data;
c. a data memory configured to store the data used in the iSCSI processor and store or retrieve said IP packets or said TCP/IP session information;
d. an instruction memory configured to store a plurality of instructions used in the iSCSI processor, the instructions related to the data;
e. an instruction fetch mechanism configured to fetch said instructions;
f. an instruction decoder configured to decode said instructions;
g. an instruction sequencer configured to sequentially organize said instructions;
h. an iSCSI session database memory configured to store the state of the session on which said data is transported, said session database memory also for recording progress of RDMA for those of said data that use RDMA;
i. an iSCSI session database memory controller configured to control said iSCSI session database memory;
j. an interface coupled to a host processor, said interface configured to receive the instructions;
said iSCSI processor coupled to a host system, said host system comprising one or more host processors and host buffers, said iSCSI processor chip further comprising:
an iSCSI processing engine for offloading iSCSI processing from said one or more host processors to perform iSCSI data transfers;
an execution resource for generating or processing said TCP/IP header and field information and for performing TCP/IP operations on said IP packets;
an RDMA execution resource for generating or processing said RDMA header and field information and for performing RDMA operations on said IP packets that require RDMA operations;
and
a transmit engine or a receive engine, or a combination thereof, coupled to one of said one or more host processors for retrieving said data from said host buffers or performing direct data placement or zero copy in said host buffers without substantial intervention by said host processor.

14 more elements

Dependencies are inferred from the claim text.

IP storage processor and engine therefor using RDMA

Life of this patent

IP storage processor and engine therefor using RDMA

Life of this patent

Abstract

Claims55