Plurality of network routers for performing collecttive operations and accelerator system including the network routers

An accelerator system comprising:

• a plurality of accelerators, each of which includes a network router configured to perform a collective operation,
• wherein each network router comprises:
• a receiver configured to receive a first input packet from a first network router along a first direction, to receive a second input packet from a second network router along a second direction, and to output one of the first input packet and the second input packet as a collective packet;
• a network controller configured to receive the collective packet output from the receiver and to output the collective packet through a first path or a second path based on a packet type of the collective packet;
• a buffer circuit configured to receive the collective packet transmitted through the second path from the network controller and to store the collective packet in a manner distinguishable according to the packet type of the collective packet;
• and
• a reduce operation circuit configured to receive the collective packet stored in the buffer circuit and to perform a reduce operation using the received collective packet,
• wherein the plurality of accelerators are interconnected in a one-dimensional torus topology.

A network router comprising:

• a receiver configured to receive a first input packet in a first direction, to receive a second input packet in a second direction, and to output one of the first input packet or the second input packet as a collective packet;
• a network controller configured to receive the collective packet from the receiver and to output the collective packet through a first path or a second path based on a packet type of the collective packet;
• a buffer circuit configured to receive the collective packet transmitted through the second path from the network controller and to store the collective packet in one or more distinct buffers according to the packet type;
• and
• a reduce operation circuit configured to receive the collective packet stored in the buffer circuit and perform a reduce operation using the received collective packet.

A network router comprising:

• a first router circuit configured to receive a first input packet along a first direction and output a first output packet along the first direction;
• and
• a second router circuit configured to receive a second input packet along a second direction and output a second output packet along the second direction,
• wherein the first router circuit comprises:
• a first receiver configured to receive the first input packet and output the first input packet as a first collective packet;
• a first network controller configured to receive the first collective packet output from the first receiver and to output the first collective packet through a first path or a second path based on a packet type of the first collective packet;
• a first buffer circuit configured to receive and store the first collective packet transmitted through the second path from the first network controller in one or more distinct first buffers according to the packet type of the first collective packet;
• and
• a first reduce operation circuit configured to receive the first collective packet stored in the first buffer circuit and perform a first reduce operation using the received first collective packet, and
• wherein the second router circuit comprises:
• a second receiver configured to receive the second input packet and output the second input packet as a second collective packet;
• a second network controller configured to receive the second collective packet output from the second receiver and to output the second collective packet through a third path or a fourth path based on a packet type of the second collective packet;
• a second buffer circuit configured to receive and store the second collective packet transmitted through the fourth path from the second network controller in one or more distinct second buffers according to the packet type of the second collective packet;
• and
• a second reduce operation circuit configured to receive the second collective packet stored in the second buffer circuit and perform a second reduce operation using the received second collective packet.