US 7,609,636 B1Granted

System and method for infiniband receive flow control with combined buffering of virtual lanes and queue pairs

AssigneeSun Microsystems, Inc.

InventorJames A. Mott

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system and method for implementing flow control, at the link and/or transport layers, for InfiniBand receive traffic. A shared memory structure may be used for combined queuing of virtual lane and queue pair traffic. Each virtual lane is allocated a number of memory buffers; a packet is dropped if buffering it would cause its virtual lane to exceed its allocation of buffers. For each active queue pair, a linked list of buffers is maintained in the structure. Each queue pair is dedicated zero or more of its virtual lane's buffers, and may also use a set of buffers shared among multiple queue pairs. Thresholds are established in a queue pair's dedicated set of buffers and/or the shared set of buffers. As each threshold number of buffers is used, a queue pair can advertise fewer message credits. RNR-NAKs are issued when no more buffers are available to a queue pair.

Claims32

Dependent claims

A method of flow controlling InfiniBand receive traffic, comprising:

maintaining a single memory structure for queuing InfiniBand traffic received via multiple virtual lanes and multiple queue pairs;
identifying a first packet payload received via a first virtual lane and a first queue pair;
determining whether the first payload can be stored in the memory structure without exceeding a portion of the memory structure allocated to the first virtual lane;
determining whether the first payload can be stored in the memory structure without exceeding a portion of the memory structure allocated to the first queue pair;
if storing the first payload in the memory structure would exceed said portion of the memory structure allocated to the first queue pair, determining whether the first queue pair is enabled to use a shared portion of the memory structure to store payloads of packets received via the first queue pair;
and
maintaining a second memory configured to store, for each of the multiple queue pairs that is active, one or more parameters associated with operation of said queue pair, wherein said parameters include:
a maximum number of message credits advertisable by said queue pair;
a maximum number of memory structure buffers dedicated to storing payloads of packets received via said queue pair;
an indicator configured to indicate whether said queue pair is enabled to use a set of shared memory structure buffers;
and
a number of shared memory structure buffers in said set of shared memory structure buffers, wherein said shared memory structure buffers are available for use by said queue pair to store payloads of packets received via said queue pair if:
said queue pair has used said maximum number of memory structure buffers;
and
said indicator indicates that said queue pair is enabled to use said set of shared memory structure buffers;
and
a maximum number of message credits advertisable by said queue pair when said queue pair starts using said shared memory structure buffers.

14 more elements

A computer readable medium storing instructions that, when executed by a computer, cause the computer to perform a method of flow controlling InfiniBand receive traffic, the method comprising:

maintaining a single memory structure for queuing InfiniBand traffic received via multiple virtual lanes and multiple queue pairs;
identifying a first packet payload received via a first virtual lane and a first queue pair;
determining whether the first payload can be stored in the memory structure without exceeding a portion of the memory structure allocated to the first virtual lane;
determining whether the first payload can be stored in the memory structure without exceeding a portion of the memory structure allocated to the first queue pair;
if storing the first payload in the memory structure would exceed said portion of the memory structure allocated to the first queque pair, determining whether the first queue pair is enabled to use a shared portion of the memory structure to store payloads of packets received via the first queue pair;
and
maintaining a second memory configured to store, for each of the multiple queue pairs that is active, one or more parameters associated with operation of said queue pair, wherein said parameters include:
a maximum number of message credits advertisable by said queue pair;
a maximum number of memory structure buffers dedicated to storing payloads of packets received via said queue pair;
an indicator configured to indicate whether said queue pair is enabled to use a set of shared memory structure buffers;
and
a number of shared memory structure buffers in said set of shared memory structure buffers, wherein said shared memory structure buffers are available for use by said queue pair to store payloads of packets received via said queue pair if:
said queue pair has used said maximum number of memory structure buffers;
and
said indicator indicates that said queue pair is enabled to use said set of shared memory structure buffers;
and
a maximum number of message credits advertisable by said queue pair when said queue pair starts using said shared memory structure buffers.

14 more elements

A method of flow controlling InfiniBand receive traffic, comprising:

maintaining a single memory structure for queuing InfiniBand traffic received via multiple virtual lanes and multiple queue pairs;
maintaining a second memory configured to store, for each of the multiple queue pairs that is active, one or more parameters associated with operation of said queue pair, wherein said parameters include a maximum number of message credits advertisable by said queue pair;
identifying a first packet payload received via a first virtual lane and a first queue pair;
determining whether the first payload can be stored in the memory structure without exceeding a portion of the memory structure allocated to the first virtual lane;
determining whether the first payload can be stored in the memory structure without exceeding a portion of the memory structure allocated to the first queue pair;
and
if storing the first payload in the memory structure would exceed said portion of the memory structure allocated to the first queue pair, determining whether the first queue pair is enabled to use a shared portion of the memory structure to store payloads of packets received via the first queue pair.

4 more elements

A computer readable medium storing instructions that, when executed by a computer, cause the computer to perform a method of flow controlling InfiniBand receive traffic, the method comprising:

maintaining a single memory structure for queuing InfiniBand traffic received via multiple virtual lanes and multiple queue pairs;
maintaining a second memory configured to store, for each of the multiple queue pairs that is active, one or more parameters associated with operation of said queue pair, wherein said parameters include a maximum number of message credits advertisable by said queue pair;
identifying a first packet payload received via a first virtual lane and a first queue pair;
determining whether the first payload can be stored in the memory structure without exceeding a portion of the memory structure allocated to the first virtual lane;
determining whether the first payload can be stored in the memory structure without exceeding a portion of the memory structure allocated to the first queue pair;
and
if storing the first payload in the memory structure would exceed said portion of the memory structure allocated to the first queue pair, determining whether the first queue pair is enabled to use a shared portion of the memory structure to store payloads of packets received via the first queue pair.

4 more elements

A method of avoiding locking in receive InfiniBand queues, the method comprising:

maintaining a single memory structure for reassembling InfiniBand traffic received via multiple virtual lanes and multiple queue pairs;
maintaining a second memory configured to store, for each of the multiple queue pairs that is active, one or more parameters associated with operation of said queue pair, wherein said parameters include a maximum number of message credits advertisable by said queue pair;
identifying a first packet payload received via a first queue pair that is idle, wherein the first queue pair is considered idle if no traffic from the first queue pair is stored in said single memory structure;
for each other queue pair for which traffic from said queue pair is stored in said single memory structure, determining whether sufficient space in the single memory structure is reserved for reassembling said traffic;
and
storing the first packet payload in said single memory structure only if sufficient space in the single memory structure is available for reassembling said traffic.

3 more elements

An apparatus for flow controlling received InfiniBand traffic, comprising:

a single memory structure configured to queue payloads of InfiniBand traffic received via multiple virtual lanes and multiple queue pairs;
a second memory configured to store, for each of the multiple queue pairs that is active, one or more parameters associated with operation of said queue pair, wherein said parameters include:
a maximum number of message credits advertisable by said queue pair;
a maximum number of memory structure buffers dedicated to storing payloads of packets received via said queue pair;
an indicator configured to indicate whether said queue pair is enabled to use a set of shared memory structure buffers;
a number of shared memory structure buffers in said set of shared memory structure buffers, wherein said shared memory structure buffers are available for use by said queue pair to store payloads of packets received via said queue pair if:
said queue pair has used said maximum number of memory structure buffers;
and
said indicator indicates that said queue pair is enabled to use said set of shared memory structure buffers;
and
a maximum number of message credits advertisable by said queue pair when said queue pair starts using said shared memory structure buffers;
a resource manager configured to manage the memory structure;
a first module configured to facilitate the advertisement of virtual lane credits;
and
a second module configured to facilitate the advertisement of queue pair credits.

12 more elements

Dependencies are inferred from the claim text.

Facts

Application: 10/812,109Filed 29 Mar 2004
Granted: 27 Oct 2009
Term ends: Not estimatedThe source records don’t support an estimate.

Classification5 codes

H04L 49/9047
Buffering arrangements: including multiple buffers, e.g. buffer pools
H04L 49/90
Packet switching elements: buffering arrangements
H04L 49/9042
Buffering arrangements: separate storage for different parts of the packet, e.g. header and payload
H04L 49/9078
Intermediate storage in different physical parts of a node or terminal: using an external memory or storage device

Family

No related US applications are recorded in the indexed filing history.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key