System and method for arbitration of memory responses in a hub-based memory system

A memory module, comprising:

• a plurality of memory devices;
• and
• a memory hub coupled to the memory devices, the memory hub comprising:
• a decoder adapted to receive memory requests including local memory requests directed to memory devices connected directly to the memory hub and remote memory requests directed to memory devices coupled to other memory hubs, the decoder being operable to determine a memory request identifier associated with each memory request;
• a packet memory adapted to receive memory request identifiers and store the memory request identifiers;
• a multiplexor adapted to receive remote memory responses that are responsive to the remote memory requests and local memory responses that are responsive to the local memory requests and being operable to select either the remote memory responses or the local memory responses in response to a control signal;
• and
• arbitration control logic coupled to the multiplexor and the packet memory and being operable to determine from the memory request identifiers stored in the packet memory the recency of the memory requests corresponding to the received remote memory responses and the local memory responses and to generate the control signal to control selection of which memory response to output based on the determination.

A memory system, comprising:

• a memory hub controller;
• a plurality of memory modules, each memory module being coupled to adjacent memory modules through respective high-speed links, at least one of the memory modules being coupled to the memory hub controller through a respective high-speed link, and each memory module comprising:
• a plurality of memory devices;
• and
• a memory hub coupled to the memory devices, the memory hub comprising,
• a decoder adapted to receive memory requests including local memory requests directed to memory devices connected directly to the memory hub and remote memory requests directed to memory devices coupled to other memory hubs, the decoder being operable to determine a memory request identifier associated with each memory request;
• a packet memory adapted to receive memory request identifiers and store the memory request identifiers;
• a multiplexor adapted to receive remote memory responses that are responsive to the remote memory requests and local memory responses that are responsive to the local memory requests and being operable to select either the remote memory responses or the local memory responses in response to a control signal;
• and
• arbitration control logic coupled to the multiplexor and the packet memory and being operable to determine from the memory request identifiers stored in the packet memory the recency of the memory requests corresponding to the received remote memory responses and the local memory responses and to generate the control signal to control selection of which memory response to output based on the determination.

A computer system, comprising:

• a processor;
• a system controller coupled to the processor, the system controller including a memory hub controller;
• an input device coupled to the processor through the system controller;
• an output device coupled to the processor through the system controller;
• a storage device coupled to the processor through the system controller;
• a plurality of memory modules, each memory module being coupled to adjacent memory modules through respective high-speed links, at least one of the memory modules being coupled to the memory hub controller through a respective high-speed link, and each memory module comprising:
• a plurality of memory devices;
• and
• a memory hub coupled to the memory devices and coupled to the corresponding high-speed links, the memory hub including,
• a decoder adapted to receive memory requests including local memory requests directed to memory devices connected directly to the memory hub and remote memory requests directed to memory devices coupled to other memory hubs, the decoder being operable to determine a memory request identifier associated with each memory request;
• a packet memory adapted to receive memory request identifiers and store the memory request identifiers;
• a multiplexor adapted to receive remote memory responses that are responsive to the remote memory requests and local memory responses that are responsive to the local memory requests and being operable to select either the remote memory responses or the local memory responses in response to a control signal;
• and
• arbitration control logic coupled to the multiplexor and the packet memory and being operable to determine from the memory request identifiers stored in the packet memory the recency of the memory requests corresponding to the received remote memory responses and the local memory responses and to generate the control signal to control selection of which memory response to output based on the determination.

In a memory system including a plurality of memory modules, each memory module including a memory hub coupled to memory devices, a method of processing and forwarding memory responses in the memory hub of each memory module, comprising:

• receiving memory requests, each having a memory request identifier, the memory requests including local memory requests directed to memory devices connected to the memory hub and remote memory requests directed to memory devices coupled to memory hubs in other memory modules;
• storing the memory request identifiers;
• storing local memory responses received from the memory devices in response to the local memory requests;
• storing remote memory responses received from the other memory modules in response to the remote memory requests;
• applying in at least one hub an arbitration algorithm based on the ages of the stored memory request identifiers to determine an order in which the stored local and remote memory responses are forwarded;
• and
• forwarding the local and remote memory responses upstream according to the determined order.