Artificial intelligence operation system and method thereof

An AI operation system comprising:

• a plurality of operators;
• a host configured to merge nodes constituting a specific attention layer in a transformer model, preprocess specific matrix data among data of the merged nodes to convert GEMV into GEMM, distribute the preprocessed data and non-preprocessed data to the plurality of operators to perform GEMM in parallel in the plurality of operators, and add and normalize operation results of the plurality of operators;
• and
• a memory configured to store the preprocessed data and the non-preprocessed data,
• wherein each of the plurality of operators receives row/column-direction data based on the distributed data and performs an addition operation, receives row-direction data based on the distributed data and performs a multiplication operation on the received row-direction data and a value obtained by performing the addition operation, performs an exponent operation on a value obtained by performing the multiplication operation, and performs a GEMM operation in parallel by performing a cumulative multiplication operation on values obtained by performing the exponent operation.

An AI operation method comprising:

• merging, by a host, nodes constituting a specific attention layer in a transformer model;
• preprocessing, by the host, specific matrix data among data of the merged node;
• distributing, by the host, the preprocessed data and non-preprocessed data to a plurality of operators;
• performing, by the plurality of operators, a parallel operation using the distributed data;
• and
• adding and normalizing, by the host, operation results of the plurality of operators.