Neural processing device and load/store method of neural processing device

A neural processing device comprising:

• at least one neural processor, each of which comprising at least one neural core;
• a first shared memory shared between the at least one neural processor;
• and
• a global interconnection comprising a data channel configured to transmit data between the at least one neural processor and the shared memory,
• wherein each of the neural core comprises:
• a processing unit configured to perform a two-dimensional matrix calculation;
• a private memory;
• and
• a load-store unit (LSU) configured to:
• perform memory access operations between the private memory and at least one shared memory,
• load data from the second memory to the first memory for a load instruction or a standby load instruction, and
• store data from the first memory to the second memory for the store instruction or the standby store instruction,
• wherein the memory access operations include a main memory access operation for a current processing operation that is performed by the processing unit, and a standby memory access operation for a standby processing operation that is performed by the processing unit after the current processing operation,
• wherein the standby memory access operation is performed by using a bandwidth that is not used by the main memory access operation within a bandwidth of the data channel,
• wherein the main memory access operation includes a main load instruction and a main store instruction, and
• wherein the standby memory access operation includes the standby load instruction and the standby store instruction.

A memory access method of a neural processing device including a load-store unit (LSU), an activation LSU, a processing unit and an activation buffer, comprising:

• performing, by the LSU, a main memory access operation to load a first data for a first work from a second memory to a first memory, the first data including a first input activation and a first weight and the first work being a two-dimensional matrix calculation of a first layer of the neural network;
• temporarily storing, by the LSU, the first input activation in the activation buffer;
• receiving, by the processing unit, the first input activation from the activation buffer and the first weight;
• generating, by the processing unit, a first output activation by performing the two-dimensional matrix calculation of the first work with the first input activation and the first weight;
• storing the first output activation in the activation buffer;
• transmitting, by the activation LSU, the first input activation from the first memory to the activation buffer and transmitting the first output activation from the activation buffer to the first memory;
• performing, by the LSU, a standby memory access operation to load a second data for a second work to be executed after the first work, the second data including a second input activation and a second weight and the second work being a two-dimensional matrix calculation of a second layer of the neural network;
• temporarily storing, by the LSU, the second input activation in the activation buffer;
• receiving, by the processing unit, the second input activation from the activation buffer and the second weight;
• and
• generating, by the processing unit, a second output activation by performing the two-dimensional matrix calculation of the second work with the second input activation and the second weight,
• wherein the standby memory access operation is performed by using a bandwidth that is not used by the main memory access operation within a bandwidth of a data channel.