Managing buffered communication between cores

A method for communicating among multiple sets of multiples cores, the method comprising:

• buffering messages in first buffer associated with a first set of multiple cores;
• buffering messages in a second buffer associated with a second set of multiple cores;
• and
• transferring messages over communication circuitry from cores not in the first set to the first buffer, and to transferring messages from cores not in the second set to the second buffer;
• wherein a first core of the first set sends messages corresponding to multiple types of instructions to a second core of the second set through the communication circuitry, where at least one first type of instruction corresponds to a first operation that is performed by the second core without requiring any instructions to be transmitted through the communication circuitry to complete the first operation, and at least one second type of instruction corresponds to a second operation that is performed by the second core requiring at least one message to be transmitted through the communication circuitry to complete the second operation, the at least one message including an acknowledgement associated with an instruction of the first type, the instruction of the first type having been initiated in response to the second operation;
• and
• wherein the second buffer is large enough to store a maximum number of instructions of the second type that are allowed to be outstanding from cores in the first set at the same time, and still have enough storage space for one or more instructions of the first type.