Credit based flow control scheme over virtual interface architecture for system area networks

A method of receiving data at a local endpoint system from a remote endpoint system across a network, the local endpoint system including a plurality of work queues for posting data transfer requests, one or more registered send and receive buffers, and one or more application receive buffers, the method comprising the steps of:

• performing the following steps if there is any data that was received from the remote endpoint system and stored in the registered receive buffers:
• copying the data from the registered receive buffers to one of the application receive buffers;
• making available or freeing any registered receive buffers that were copied into the application receive buffers;
• and
• updating a number of receive credits based on the step of freeing or making available any registered receive buffers.

A method of receiving data at a local endpoint system from a remote endpoint system across a network, the local endpoint system including a plurality of work queues for posting data transfer requests, one or more registered send and receive buffers, and one or more application receive buffers, the method comprising the steps of:

• receiving a packet;
• determining whether the packet is a data packet;
• performing the following steps if the packet is a data packet:
• polling for any additional packets that have been received by the local endpoint system;
• copying the data for all the received packets from the registered receive buffers to one or more application buffers;
• freeing or making available the registered buffers that have been copied to application buffers;
• and
• updating the number of receive credits based on the additional available receive buffers.

A method of receiving data at a local endpoint system from a remote endpoint system across a network, the local endpoint system including a plurality of work queues for posting data transfer requests, one or more registered send and receive buffers, and one or more application receive buffers, the method comprising the steps of:

• receiving a packet;
• determining if the received packet is a credit request packet;
• if the received packet is a credit request packet, then performing the following steps:
• determining the number of available or free registered receive buffers;
• sending a credit response packet to the remote endpoint system if the number of free registered receive buffers at the local endpoint system is greater than a threshold value, otherwise marking a pending credit response if the number of free receive buffers is not greater than the threshold value.

A method of sending data from a local endpoint system to a remote endpoint system across a network, the local endpoint system including a plurality of work queues for posting data transfer requests, the method comprising the steps of:

• fragmenting data to be sent into two or more data packets if necessary;
• performing the following steps until all packets have been sent:
• a) determining if a sufficient number of send credits is available at the local endpoint system;
• b) sending a data packet from the local endpoint system over the network if a sufficient number of send credits is available, and adjusting the number of send credits based on the sending of the data packet;
• and
• c) otherwise, if a sufficient number of send credits is not available at the local endpoint system, sending a credit request packet from the local endpoint system to the remote endpoint system and waiting for a credit response packet from the remote endpoint system before sending a data packet.

A first endpoint system for communicating with a second endpoint system across a network, the first endpoint system comprising:

• an interface consumer, including:
• an application;
• a plurality of send and receive buffers;
• a transport service provider coupled to the application, the transport service provider providing an end-to-end credit-based flow control for transferring data between the first and second endpoints;
• an interface user agent coupled to the transport service provider;
• an interface provider coupled to the interface user agent, the interface provider including:
• a kernel agent;
• a plurality of work queues;
• and
• a network interface controller coupled to the kernel agent and the work queues;
• said interface user agent issuing one or more control commands to the kernel agent to establish a connection between the first and second endpoint systems across the network and posting data transfer requests to the work queues in response to commands from the transport service provider;
• and
• the network interface controller processing the data transfer requests by transferring data between the send and receive buffers and the network.

A first endpoint system for communicating with a second endpoint system across a network, said first endpoint system comprising:

• an application, the application including send and receive application buffers;
• an operating system communications facility coupled to the application and adapted to receive commands from the application;
• a transport service provider coupled to the communications facility and adapted to receive commands from the operating system communications facility, the transport service provider providing an end-to-end credit-based flow control for data transfer across the network;
• a network interface controller coupled to the network;
• a plurality of work queues coupled to the network interface controller for posting data transfer requests thereto;
• a user agent coupled to the send and receive buffers, the operating system communications facility and the network interface controller, the user agent posting data transfer requests to the work queues in response to commands from the operating system communications facility, the network interface controller processing the posted data transfer requests by transferring data between the send and receive buffers and a network.

A local endpoint system for communicating to a remote endpoint system over a network, the local endpoint system comprising:

• a host processor;
• one or more work queues for posting data transfer requests;
• one or more registered send buffers;
• one or more registered receive buffers;
• a network interface controller coupled to the host processor, the work queues, the buffers and the network, the network interface controller processing the posted data transfer requests by transferring data between the registered buffers and the network;
• and
• the host processor being programmed to control the transfer of data performed by the network interface controller based on a credit-based flow control scheme.

A method of transferring data between first and second endpoint systems over a system area network, each endpoint system including registered send and receive buffers for sending and receiving data, comprising:

• determining if a sufficient number of send credits is available at the first endpoint system;
• sending a data packet from the first endpoint system to the second endpoint system over the network if a sufficient number of send credits is available at the first endpoint system;
• and
• otherwise, if a sufficient number of send credits is not available at the first endpoint system, sending a credit request packet from the first endpoint system to the second endpoint system and waiting for a credit response packet from the second endpoint system before sending a data packet;
• and
• the second endpoint system identifying the number of available receive buffers at the second endpoint system, the second endpoint system sending a credit response packet to the first endpoint system in response to the credit request packet only if the number of available receive buffers at the second endpoint system is greater than a threshold value.