Fibre channel arbitrated loop bufferless switch circuitry to increase bandwidth without significant increase in cost

A bufferless switch for coupling to a plurality of Fibre Channel Arbitrated Loops comprising:

• a fairness token bus;
• a protocol bus;
• a memory;
• a crossbar switch;
• and
• Fibre Channel Arbitrated Loop interface port circuits structured to use the OPEN POINT TO POINT and RECEIVER READY primitives of the Fibre Channel Arbitrated Loop protocol for holding-back flow control to eliminate the need for a buffer;
• and wherein said ports and crossbar switch are structured to provide multiple simultaneous loop tenancies and dual simplex communication across said switch,
• and wherein each said Fibre Channel Arbitrated Loop interface port circuit includes a local bypass data path which can be switched to keep primitives and data frames generated on a local Fibre Channel Arbitrated Loop coupled to said port routed so as to stay on said local Fibre Channel Arbitrated Loop such that local loop tenancies on each local Fibre Channel Arbitrated Loop coupled to a port which do not involve transmission of data or primitives from one said Fibre Channel Arbitrated Loop interface port circuit to another can occur simultaneously,
• and wherein each said Fibre Channel Arbitrated Loop interface port circuit includes local learning memory means for storing a routing table and scoreboard table, and functioning to fully cache the 8-bit addresses of all nodes on each said local Fibre Channel Arbitrated Loop coupled to a port along with the ID of the port coupled to each node in said routing table and the busy, available or no privilege status of each other Fibre Channel Arbitrated Loop interface port circuit and whether each busy Fibre Channel Arbitrated Loop interface port circuit is available for dual simplex communications in said scoreboard table,
• and wherein each said Fibre Channel Arbitrated Loop interface port circuit is structured to write the contents of each said routing table by learning either from watching local traffic in said Fibre Channel Arbitrated Loop interface port circuit bound for local destination nodes or remote traffic leaving said Fibre Channel Arbitrated Loop interface port circuit and crossing said crossbar switch bound for remote destination nodes or by conducting an active discovery process,
• and wherein each said Fibre Channel Arbitrated Loop interface port circuit is structured to write the contents of said scoreboard table by learning from watching messages posted on said protocol bus,
• and wherein each said Fibre Channel Arbitrated Loop interface port circuit is structured to use destination addresses in OPEN POINT TO POINT primitives transmitted by nodes on said local Fibre Channel Arbitrated Loop as search keys to search said routing table, to determine whether the destination node is local or remote and to which said Fibre Channel Arbitrated Loop interface port circuit it is connected and, if local, to switch said local bypass to keep the loop tenancy confined to said local Fibre Channel Arbitrated Loop, or, if remote, to generate a connection request message to said Fibre Channel Arbitrated Loop interface port circuit coupled to said destination node,
• and wherein each said Fibre Channel Arbitrated Loop interface port circuit is structured to receive such, connection requests and respond by arbitrating for control of its local Fibre Channel Arbitrated Loop and pick a data path through said crossbar switch and generate a reply message naming the data path through said crossbar switch to use in exchanging data frames and primitives,
• and wherein each said Fibre Channel Arbitrated Loop interface port circuit is also structured to respond to receipt or generation of said reply message by generating commands to establish said data path named in said reply message through said crossbar switch,
• and wherein each said Fibre Channel Arbitrated Loop interface port circuit is also structured to circulate a fairness token on said fairness token bus coupled to all such ports and to use said fairness token to increase its priority level of access privilege to busy nodes when the fairness token is in possession of the Fibre Channel Arbitrated Loop interface port circuit so that no Fibre Channel Arbitrated Loop interface port circuit can be starved, and then to circulate said fairness token to a neighboring Fibre Channel Arbitrated Loop interface port circuit such that all Fibre Channel Arbitrated Loop interface port circuits eventually get said fairness token.

A bufferless switch for coupling one or more pairs of nodes on a plurality of Fibre Channel Arbitrated Loops so as to allow simultaneous loop tenancies, said switch comprising:

• a crossbar switch means for creating multiple data paths to carry data of one or more loop tenancies between one or more pairs of nodes on one or more Fibre Channel Arbitrated Loops coupled to said bufferless switch;
• a plurality of Fibre Channel Arbitrated loop interface port means, each coupled to a different Fibre Channel Arbitrated Loop, each for cooperating with said crossbar switch to allow a node on the Fibre Channel Arbitrated Loop to which said port means is coupled to establish a loop tenancy with a node on another Fibre Channel Arbitrated Loop coupled to another port means, and for using the OPEN POINT TO POINT and RECEIVER READY primitives of the Fibre Channel Arbitrated Loop protocol for holding-back flow control to eliminate the need for a buffer large enough to store an entire frame anywhere in said bufferless switch;
• and wherein said port means and crossbar switch means are also cooperating to provide multiple simultaneous loop tenancies during switch mode or, upon receiving a command to establish hub mode, to implement a hub mode by coupling all said Fibre Channel Arbitrated Loops together through said crossbar switch means and said port means as one big Fibre Channel Arbitrated Loop;
• and wherein said port means and crossbar switch means can be switched back and forth between hub mode and switch mode by going through an intermediate locked mode;
• a protocol bus coupling all said port means together, said protocol bus being controlled by said port means to be multiplexed using multiplexing scheme.

A Fibre Channel Arbitrated Loop Switch, comprising:

• a fairness token bus;
• a protocol bus;
• a backplane data path capable of supporting multiple bidirectional data paths;
• a plurality of switch chip circuits, each comprising:
• a memory containing routing table data;
• a memory containing scoreboard table data;
• a learning half bridge front end circuitry which includes an FCAL port comprised of an input port and an output port for coupling to the transmit and receive lines of a Fibre Channel Arbitrated Loop or a single Node Loop port which is part of a computer or a computer peripheral device, said computer or computer peripheral device referred to herein as a node, each learning half bridge including circuitry to implement the Fibre Channel Arbitrated Loop protocol to communicate with nodes coupled to said FCAL port and to receive connection requests from said protocol bus, and including circuitry coupled to said FCAL port to arbitrate for control of said Fibre Channel Arbitrated Loop coupled to said FCAL port, if necessary, using said Fibre Channel Arbitrated Loop Protocol when a connection request is received at said switch chip circuit from another switch chip circuit, and to send message data on said protocol bus indicating the status of said FCAL port as busy or idle and to learn from message traffic on said protocol bus which other switch chip circuits are coupled to other nodes coupled to said Fibre Channel Arbitrated Loop Switch and store that data in said memory containing said routing table data and to learn from said message traffic on said protocol bus which said FCAL ports of other switch chip circuits are busy and which are idle and store that data in said memory storing scoreboard table data;
• streaming back end crossbar switch circuitry coupled to said backplane implementing a slice of a distributed crossbar switch and functioning to connection requests from said learning half bridge circuits to use Open Point to Point and Receiver Ready primitives received or generated at said FCAL ports for flow control in loop tenancies so that Fibre Channel Arbitreated Loop frames of data and primitives can be streamed across bidirectional channels through said backplane to implement multiple loop tenancies between multiple pairs of source and destination nodes without ever dropping a frame and without the need for any buffer memory big enough to store an entire Fibre Channel Arbitrated Loop frame and without the need for any error recovery protocols;
• circuitry coupling said port circuit to said fairness token bus to send and receive fairness tokens such that said fairness token is circulated to all said switch chips;
• circuitry coup led to said protocol bus to determine from said routing table which said switch chip circuit is coupled to a destination node named in an Open Point to Point primitive received at said FCAL port and to determine if said FCAL port of said switch chip circuit coupled to said destination node is busy or idle from said scoreboard table data, and if idle, to use said protocol bus to send a connection request to said switch chip circuit coupled to said destination node.

A Fibre Channel Arbitrated Loop Switch, comprising:

• a protocol bus;
• a backplane data path circuit capable of supporting multiple, parallel point-to-point, differential, bidirectional data paths, said backplane data path circuit including a strobe circuit;
• a plurality of switch chip circuits, each implementing a plurality of Fibre Channel Arbitrated Loop ports (hereafter FCAL ports), each having a unique identification code, said switch chip circuit comprising:
• a memory containing routing table data;
• a plurality of serializer/deserializer (hereafter SERDES) circuits, each having a transmit and a receive port for coupling to a Fibre Channel Arbitrated Loop or a single Node Loop port and functioning to receive differentially encoded 8b/10 serial data, demultiplex it into aligned characters and recover a receive clock therefrom, and to receive aligned characters for transmission and multiplex them into a differentially encoded 8b/10 serial output stream;
• a plurality of elastic buffers, each coupled to a different one of said SERDES circuit, each for absorbing the differences between transmit and receive data rates and retiming received data so as to be compatible with transmit data output by said SERDES circuit;
• a plurality of loop port state machines, each coupled to a different one of said elastic buffers and each functioning to implement a routing algorithm which uses the Open Point To Point and Receiver Ready primitives of a Fibre Channel Arbitrated Loop protocol to implement flow control so frames are never dropped or lost and never have to be buffered in said switch, said routing algorithm including the ability to arbitrate for control of the Fibre Channel Arbitrated Loop coupled to the SERDES circuit to said loop port state machine is coupled when data is received from said backplane requiring transmission on said Fibre Channel Arbitrated Loop coupled to the SERDES circuit to said loop port state machine is coupled, and once control is achieved, for forwarding any Open Point to Point or other primitives and data frames onto said Fibre Channel Arbitrated Loop coupled to the SERDES circuit to said loop port state machine is coupled, and loop port state machine also for sending message data on said protocol bus indicating the status of said SERDES circuit as busy or idle, said loop port state machine also functioning to learn which nodes are coupled to which SERDES circuits of each switch chip circuit by either by a discovery process or by trapping Open Point to Point primitives and storing the data so learned in a portion of said memory containing said routing table data, and said loop port state machine using said routing table data stored in said memory during said routing algorithm to determine which said switch chip circuit is coupled to a destination node named in an Open Point to Point primitive received from said backplane or from said elastic buffer to which said loop port state machine is coupled and to send a connection request thereto;
• a port multiplexer circuit having a plurality of multiplexer circuits each controlled by a separate port multiplexer state machine and each coupled by a bidirectional data path input coupled to a different one of said loop port state machines and having a plurality of bidirectional output data paths coupled to said backplane data path circuit, said port multiplexer state machines controlling switching by said port multiplexer circuits such that local loop tenancies between nodes on the Fibre Channel Arbitrated Loops coupled to one of the plurality of SERDES circuits within the same switch chip circuit are kept off said backplane by making a bypass connection between the two port multiplexer state machines coupled by bidirectional data paths to the loop port state machines coupled to the Fibre Channel Arbitrated Loops having the source and destination nodes of the local loop tenancy thereon, and said port multiplexer state machines controlling switching by said multiplexer circuits such that remote loop tenancies between a source node on one of the Fibre Channel Arbitrated Loops coupled to said switch chip circuit and a destination node on a Fibre Channel Arbitrated Loop coupled to another of said switch chip circuits are implemented by coupling the bidirectional data path coupled to the loop port state machine coupled to the Fibre Channel Arbitrated Loop having the source node thereon to said backplane, and wherein said port multiplexer state machines can control said bidirectional data paths to implement full duplex or dual-simplex operation, said port multiplexer also containing a camp list First-In-First-Out memory (hereafter FIFO) for each of the three SERDES circuits in said switch chip circuit, said FIFO containing the identification codes of remote ports that have made requests for connections but which have not yet had connections through said backplane established.

A bufferless switch to couple a plurality of Fibre Channel Arbitrated Loops and allow one or more simultaneous loop tenancies, comprising:

• a fairness token bus;
• a protocol bus;
• a plurality of backplane data paths timed by a strobe circuit;
• a look up table for storing routing data;
• a plurality of switch chip circuits, each coupled to said fairness token bus, said protocol bus and said plurality of backplane data paths and said strobe circuit, and each having a port for coupling to a Fibre Channel Arbitrated Loop or a single Loop Node port of a computer or peripheral device, each switch chip circuit comprising:
• a plurality of serializer/deserializer circuits (hereafter SERDES circuits);
• a plurality of elastic buffers, each coupled to a different one of said SERDES circuits;
• a plurality of loop port state machines, each coupled to a different one of said elastic buffers, and each having a transmit and receive port, and each structured to implement a predetermined routing algorithm that uses the Open Point to Point and Receiver Ready primitives of a Fibre Channel Arbitrated Loop protocol to implement holding-back flow control such that Fibre Channel Arbitrated Loop frames being transmitted between different nodes on one or more Fibre Channel Arbitrated Loops coupled to one or more of said switch chip circuits are never lost and never have to be stored in said bufferless switch;
• a port multiplexer coupled to each of said transmit and receive ports of said loop port state machines and having a plurality of ports coupled to said backplane data paths so as to form a ring of loop port state machines coupled by one or more data paths of said backplane;
• and wherein said loop port state machines control said port multiplexer to keep traffic local to a particular Fibre Channel Arbitrated Loop local by setting a bypass switch in the switch chip circuit coupled to the Fibre Channel Arbitrated Loop on which both the source node and destination node are resident, but to use said backplane data paths to connect source and destination nodes on different Fibre Channel Arbitrated Loops such that one or more loop tenancies can be simultaneously carried out over one or more backplane data paths without buffering any frames and without ever losing any frames thereby eliminating the need for recovery protocols to handle lost frame situations.

A process for routing in a bufferless switch coupled to a plurality of Fibre Channel Arbitrated Loops (hereafter FCAL) via a plurality of Fibre Channel Arbitrated Loop port circuits (hereafter FCAL port), said port circuits coupled to a protocol bus of said bufferless switch and to a backplane circuit capable of establishing a one or more data paths between said FCAL ports, each said data path designated as a channel and being bidirectional and capable of carrying primitives and data frames of a Fibre Channel Arbitrated Loop tenancy, comprising the steps:

• 1) receiving at an FCAL port designated hereafter as a source port of a loop tenancy, an Open Point to Point primitive (hereafter OPN) and zero or more Receiver Ready primitives (hereafter RRDY) from a node hereafter referred to as a source node on a FCAL coupled to a bufferless switch, and latching said primitives, and determining on which FCAL a destination node identified in said OPN resides by consulting a routing table in said switch, and wherein more than one source port may exist at any time;
• 2) for each source port, determining from said routing table data whether the destination node is on the same FCAL as the source node, and, if so, proceeding to step 3, and, if said destination node is not on the same FCAL as said source node, proceeding to step 9;
• 3) in each said source port, if said destination node designated in said OPN received from said source node is on the same FCAL as said source node, as determined by consulting said routing table data, establishing a port bypass data path in said source port of said bufferless switch which is coupled to said FCAL on which both said source and destination nodes are resident so as to keep the primitives and data frames of a loop tenancy between said source node and destination node local by passing said primitives and data frames through said bypass data path, said loop tenancy hereafter referred to as a local loop tenancy, and forwarding all primitives latched in said source port to said destination node and all subsequent primitives and data frames through said bypass data path such that said source node only transmits a frame to said destination node when an RRDY primitive is received back from said destination node and said destination node only transmits a data frame to said source node when an RRDY primitive is received from said source node, thereby eliminating the need for a buffer in said source port which is big enough to store an entire data frame and eliminating any lost data frames and the need for error recovery protocols to recover from lost data frame situations, and terminating said local loop tenancy and resetting said bypass data path and relinquishing control of said FCAL when a Close primitive is transmitted either from said source node or said destination node to the other node in said local loop tenancy;
• 4) in each FCAL port, after said bypass data path established to carry data frames and primitives between source and destination nodes involved in a local loop tenancy has been reset following conclusion of said loop tenancy, monitoring for reception of an OPN primitive from a source node on said FCAL formerly involved in said local loop tenancy, and repeating steps 1 through 4 herein, and, at any time, including during the existence of said local loop tenancy or after said local loop tenancy is terminated, monitoring for reception from a source port of a connect request on a protocol bus of said bufferless switch which coupled all said FCAL ports together, and if a connect request is received, said FCAL port which received said connect request on said protocol bus arbitrates as a destination port for control of a local FCAL coupled to said destination port to begin the process of establishing a remote loop tenancy between a source node coupled by a different, remote FCAL coupled to a source port which sent said connect request, and wherein data frame and primitive exchange between said source port coupled to said remote FCAL and a source node coupled thereto and said destination port coupled to said local FCAL and a destination node on said local FCAL being hereafter referred to as a remote loop tenancy, and wherein multiple simultaneous local and remote loop tenancies are possible between nodes resident on the FCALs coupled to said bufferless switch;
• 5) after control of said FCAL is achieved by said destination port of a remote loop tenancy, sending back a response message to said source port indicating control of said FCAL has been accomplished and designating a specific backplane channel to use in transmitting primitives and data frames of said remote loop tenancy and requesting said source port to forward any stored primitives;
• 6) generating commands in each said source port and destination port involved in said loop tenancy to establish said backplane channel identified in said response message in time to carry primitives and data frames of said remote loop tenancy, and forwarding any primitives of said remote loop tenancy stored in said source port to said destination node through said designated backplane channel and said destination port, and continuing to transmit primitives and data frames of said remote loop tenancy until said remote loop tenancy is finished, said source node only sending a data frame when an RRDY primitive is received from said destination node, and said destination node only sending a data frame to said source node when said source node sends an RRDY primitive to said destination node, thereby eliminating the need for a buffer in said bufferless switch which is big enough to store an entire data frame and eliminating the need for error recovery protocols in said bufferless switch to recover from situations wherein data frames are lost;
• 7) after a Close primitive is transmitted either by said source node or said destination node, closing said backplane channel used by said remote loop tenancy, and relinquishing control of the FCALs involved in said remote loop tenancy and sending a release frame message on said protocol bus indicating the availability for further loop tenancies of said backplane channel and said source and destination ports involved in said remote loop tenancy;
• 8) receiving said release frame message on said protocol bus, and recording in a scoreboard table, the availability of said backplane channel and said source and destination ports for further transactions, and maintaining the contents of said scoreboard table to reflect the current status of each said backplane channel and each said FCAL port of said bufferless switch by monitoring messages sent on said protocol bus;
• 9) determining the idle or busy status of an FCAL port coupled to said FCAL on which said destination node resides, referred to herein as a destination port, by checking status data stored in a scoreboard table;
• 10) if said destination port is busy, performing a step to prevent a loop tenancy from being established at least temporarily;
• 11) if said destination port is idle, sending a connect request to said destination port using said protocol bus, and repeating steps 4 through 8 until said remote loop tenancy has been completed.

A routing process carried out in a bufferless switch coupled to a plurality of Fibre Channel Arbitrated Loops (hereafter FCAL) by Fibre Channel Aribtrated Loop Ports (hereafter FCAL ports), each FCAL being coupled to one or more computers or peripheral devices through a Node Loop port (hereafter called nodes), each node having an address, and each said FCAL port coupled to the other FCAL ports through a protocol bus which carries connection requests and other messages indicating busy or idle status and a backplane circuit capable of being controlled by said FCAL ports to establish multiple simultaneous bidirectional data paths, each referred to herein as a backplane channel, to carry data frames and primitives of loop tenancies between ports, and each FCAL port having a port identification code (hereafter port ID), said bufferless switch containing a memory which can be centralized or distributed among said FCAL ports for storing routing table data and scoreboard table data, said bufferless switch for establishing one or more simultaneous loop tenancies without buffering any data frames in said bufferless switch, comprising the steps:

• learning the addresses of each node and the port ID of the FCAL port coupled to the FCAL on which the node is resident by watching traffic in local loop tenancies between nodes on the same FCAL and watching traffic between nodes on different FCALs crossing said backplane data paths or by conducting a discovery process and writing the addresses and port IDs learned in a routing table;
• learning the busy or idle status of each said FCAL port by watching message traffic on said protocol bus and recording said status information in said memory as a scoreboard table;
• at each FCAL port, receiving Open Point to Point primitives (hereafter OPN primitives) and any Receiver Ready primitives (hereafter RRDY primitives) from source nodes on the FCAL to which said FCAL port is coupled and buffering said primitives and using a destination address of a destination node in said OPN and said routing table data to determine if said destination node is on the same FCAL as said source node, and, if so, establishing a bypass data path to establish a data path not involving said backplane to carry primitives and data frames of a local loop tenancy between said source and destination nodes and forwarding said buffered OPN and any buffered RRDY primitives to said destination node and passing all primitives and data frames of said local loop tenancy between said source and destination nodes via said bypass data path, and wherein said OPN and RRDY primitives of the Fibre Channel Arbitrated Loop protocol of said local loop tenancy serve as an inherent flow control such that data frames of said local loop tenancy never need to buffered in said bufferless switch and are never lost;
• if said destination address in said OPN is the address of a destination node on a different FCAL than said source node, generating a connect request message on said protocol bus to an FCAL port coupled to an FCAL coupled to said destination node (hereafter referred to as the destination port);
• in each said FCAL destination port which is idle, arbitrating for control of said FCAL coupled to said destination node, and when control is achieved, sending back a reply message to an FCAL port which sent said connect request message (hereafter referred to as the source port) indicating that control has been achieved and instructing said source port which backplane channel to use for a remote loop tenancy to be established between said source node and said destination node via said source port and said destination port;
• generating commands in said source port and destination port to control said backplane circuit to establish said backplane channel identified in said reply message, and forwarding said buffered OPN primitive and any buffered RRDY primitives stored in said source port to said destination node via said backplane channel and said destination port, and thereafter forwarding all primitives and data frames of said remote loop tenancy between said source node and destination node, and wherein said OPN and RRDY primitives of the Fibre Channel Arbitrated Loop protocol of said remote loop tenancy serve as an inherent flow control such that data frames of said remote loop tenancy never need to buffered in said bufferless switch and are never lost.