High performance computing system for deep learning

A system comprising:

• a host processor configured to generate a learning command for a plurality of input data;
• a first accelerator configured to perform an operation on first input data of the plurality of input data based on neural network variables in response to the learning command and to generate a first operation result and a second operation result;
• a second accelerator configured to perform an operation on second input data of the plurality of input data based on neural network variables in response to the learning command and to generate a third operation result and a fourth operation result;
• a first memory node of a plurality of memory nodes configured to store the first operation result transferred directly from the first accelerator through a first data link, the plurality of memory nodes is distinguished from a host memory;
• a second memory node configured to store the second operation result transferred directly from the first accelerator through a second data link;
• and
• a third memory node configured to store the fourth operation result transferred from the second accelerator through a third data link,
• wherein the first accelerator, without interaction with the host processor and the host memory, directly accesses the first memory node through the first data link and directly accesses the second memory node through the second data link, and
• wherein the first memory node stores the third operation result transferred from the second accelerator through a fourth data link.

A system comprising:

• a host processor;
• a plurality of accelerators configured to communicate with the host processor based on a communication interface;
• and
• a plurality of memory nodes distinguished from a host memory and is connected with the plurality of accelerators through an interconnection network,
• wherein the interconnection network is between the plurality of accelerators and the plurality of memory nodes and comprises data links through which the accelerators directly access the memory nodes without interaction with the host processor and the host memory,
• wherein a first memory node of the plurality of memory nodes directly accesses a first accelerator through a first of the data links of the interconnection network and directly accesses a second accelerator of the plurality of accelerators through a second of data links of the interconnection network,
• wherein the second accelerator is different than the first accelerator, and
• wherein the first accelerator and the second accelerator directly exchange data with each other through the first memory node in which the first accelerator transfers data to the second accelerator through the first memory node, and the second accelerator transfers data to the first accelerator through the first memory node.