Head-wise token pruning method and electronic device

An electronic device, comprising:

• a memory used to store a plurality of tokens;
• and
• a processor electrically connected to the memory and used to read the tokens and execute a plurality of steps:
• computing a plurality of query matrices and a plurality of key matrices corresponding to a plurality of heads;
• computing an attention matrix according to the corresponding query matrix and key matrix for each of the heads, wherein the attention matrix corresponds to the tokens and comprises a plurality of attention values;
• computing an importance corresponding to each of the tokens according to the attention values and determining at least one pruned token in the tokens according to the importance for each of the heads, wherein the heads comprise a first head and a second head, and the at least one pruned token of the first head is different from the at least one pruned token of the second head;
• and
• omitting the at least one pruned token corresponding to each of the heads in a subsequent computation.