Tensor processor with re-routable processing elements with multiple multiplexers and a method for processing tensors with the tensor processor thereof

A method for processing a tensor having a central control module, a processing element controller, a plurality of tensor manipulation modules, a plurality of processing elements, and multiple multiplexers, wherein the method comprises:

• decoding, by the central control module, a received tensor operation instruction;
• based on the decoded instruction, selectively instructing the processing element controller by one of the plurality of tensor manipulation modules to perform a plurality of tensor operations;
• signaling the multiple multiplexers by the processing element controller to configure each of the plurality of processing elements to be arranged in a plurality of processing columns, wherein each processing element comprises local block memory including at least a first input block memory (BRAM-A), a second input block memory (BRAM-B), and an output block memory (BRAM-C), and wherein the multiple multiplexers are coupled to the plurality of processing elements and controlled by the processing element controller;
• arranging an adder, a multiplier, the first input block memory (BRAM-A) for storing a first input matrix, the second input block memory (BRAM-B) for storing a second input matrix, and the third input block memory (BRAM-C) for storing an output result matrix;
• coordinating, by the processing element controller, data transfers involving the first input block memory (BRAM-A), the second input block memory (BRAM-B), and the output block memory (BRAM-C) associated with the plurality of processing elements to perform double buffering of the first input matrix, the second input matrix and the output result matrix by alternately using one of the BRAM-A and the BRAM-B for receiving new input data, wherein, when the BRAM-A is used for receiving new input data, the BRAM-B is providing data for computation, and, when the BRAM-B is used for receiving new input data, the BRAM-A is providing data for computation, and alternately using portions of the BRAM-C for storing new results while other portions of the BRAM-C are being read out,
• wherein the plurality of processing elements function as two cascaded two-dimensional matrix multiplication arrays, and each of the plurality of processing elements has a local block memory size (T) for input data dependent on one or more factors comprising output matrix size (I, J), size of a matrix multiplication arrays (Npex, Npey), and clock cycles of computation delay of a multiplier and an adder of a processing element (Tmul, Tadd),
• wherein, for performing a selected one of the plurality of tensor operations corresponding to the received tensor operation instruction, the processing element controller is arranged to configure, in real time, the plurality of processing elements located within a single processing column of the plurality of processing columns to selectively form one of a plurality of computational structures to perform the selected one of the plurality of tensor operations;
• wherein the selected one of the plurality of tensor operations is performed by the one of a plurality of computational structures such that all intermediate computational results generated and consumed solely within the single processing column during the performance of the selected tensor operation are stored and processed using local block memory without transferring data to off-chip memory during the performing of the selected one of the plurality of tensor operations,
• wherein the plurality of computational structures comprises an adder-tree, and a multiplier adder tree, and
• wherein the plurality of tensor operations comprises Khatri-Rao product operation, Hadamard product operation, Tensor times matrix and matrix multiplication operation, Matrix inversion operation, maximum normal of each columns of a matrix or Euclidean normal of each column of a matrix, and Tensor times matrices chain (TTMc) operations.