Scalable directory based cache coherence protocol

A distributed multiprocessing computer system, comprising:

• a plurality of processors each coupled to a memory module, wherein each memory module may store data that may be accessed by any of said plurality of processors;
• a Home processor that includes a memory block and a directory for said memory block in an associated memory module;
• a plurality of Sharer processors, each containing a shared copy of said memory block in a cache coupled to the Sharer processor memory module;
• and
• a plurality of coherence controllers, wherein each coherence controller is coupled to an associated memory module, with each of said coherence controllers maintaining coherence of said shared data in the associated memory module using said directory in said Home processor, wherein each coherence controller is capable of sending and receiving messages out of order to maintain the coherence of said shared data in said memory module, and wherein said coherence controller also is capable of restoring the prior correct saved program state and resuming execution if out of order messages cause an incorrect next program state.

A distributed multiprocessing computer system, comprising:

• a plurality of processors each coupled to an associated memory module, wherein each memory module may store data that is shared between said processors;
• a first processor that includes a memory block and a directory for said memory block in an associated memory module;
• an second processor that includes a copy of said memory block in a cache memory associated with the second processor, said copy of said memory block residing exclusively in said cache memory;
• and
• a plurality of coherence controllers, wherein each coherence controller is coupled to an associated memory module, with each of said coherence controllers maintaining coherence of said shared data in the associated memory module using said directory in said first processor, wherein replacement of said copy of said memory block in said cache memory associated with the second processor requires writing said copy of said memory block to the memory module associated with the first processor and updating the first processor directory without performing a read of the first processor directory or modifying other processor memory modules in the computer system.

A distributed multiprocessing computer system, comprising:

• a plurality of processors each coupled to a memory module, wherein each memory module may store data that may be accessed by any of said plurality of processors;
• a first processor that includes a memory block and a directory for said memory block in an associated memory module;
• a plurality of second processors, each containing a shared copy of said memory block in a cache coupled to the second processor memory module;
• and
• a plurality of coherence controllers, wherein each coherence controller is coupled to an associated memory module, with each of said coherence controllers maintaining coherence of said shared data in the associated memory module using said directory in said first processor, wherein each coherence controller is capable of sending and receiving messages out of order to maintain the coherence of said shared data in said memory module, and wherein said coherence controller also is capable of restoring the prior correct saved program state and resuming execution if out of order messages cause an incorrect next program state.

A method of maintaining memory coherence in a distributed shared memory computer system including a plurality of processors, comprising the acts of:

• requesting a copy of a memory block from a Home processor to perform a write operation on the copy of the memory block;
• storing said copy of said memory block exclusively in a cache memory associated with an Owner processor;
• updating a coherence directory for said memory block in said Home processor to indicate the write operation on the copy of said memory block in said cache memory associated with the Owner processor;
• replacing said copy of said memory block in said cache memory associated with the Owner processor, wherein said replacing requires writing said copy of said memory block to a memory module associated with the Home processor and updating of the Home processor coherence directory without performing a read of the Home processor directory or modifying other processor memory in the computer system.

A method of maintaining memory coherence in a distributed shared memory computer system including a plurality of processors, comprising:

• requesting a copy of a memory block from a Home processor to perform a read operation on the copy of the memory block, said copy of said memory block stored in each of a plurality of cache memories each associated with a Sharer processor;
• updating a coherence directory for the copy of the memory block in said Home processor to indicate the read operation on the copy of the memory block by sending and receiving messages out of order;
• and
• restoring a prior saved program state and resuming execution if the out of order messages cause an incorrect next program state.

A method of maintaining memory coherence in a distributed shared memory computer system including a plurality of processors, comprising the acts of:

• requesting a copy of a memory block from a first processor to perform a write operation on the copy of the memory block;
• storing said copy of said memory block exclusively in a cache memory associated with a second processor;
• updating a coherence directory for said memory block in said first processor to indicate the write operation on the copy of said memory block in said cache memory associated with the second processor;
• replacing said copy of said memory block in said cache memory associated with the second processor, wherein said replacing requires writing said copy of said memory block to a memory module associated with the first processor and updating of the first processor coherence directory without performing a read of the first processor directory or modifying other processor memory in the computer system.

A method of maintaining memory coherence in a distributed shared memory computer system including a plurality of processors, comprising:

• requesting a copy of a memory block from a first processor to perform a read operation on the copy of the memory block, said copy of said memory block stored in each of a plurality of cache memories each associated with a second processor;
• updating a coherence directory for the copy of the memory block in said first processor to indicate the read operation on the copy of the memory block by sending and receiving messages out of order;
• and
• restoring a prior saved program state and resuming execution if the out of order messages cause an incorrect next program state.

A distributed multiprocessing computer system, comprising:

• a plurality of processors each coupled to an associated memory module, wherein each memory module may store data that is shared between said processors;
• a Home processor that includes a memory block and a directory for said memory block in an associated memory module;
• an Owner processor that includes a copy of said memory block in a cache memory associated with the Owner processor, said copy of said memory block residing exclusively in said cache memory;
• a plurality of coherence controllers, wherein each coherence controller is coupled to an associated memory module, with each of said coherence controllers maintaining coherence of said shared data in the associated memory module using said directory in said Home processor, wherein replacement of said copy of said memory block in said cache memory associated with the Owner processor requires writing said copy of said memory block to the memory module associated with the Home processor and updating the Home processor directory without performing a read of the Home processor directory or modifying other processor memory modules in the computer system;
• and
• a disk drive coupled to each of said plurality of processors.