Mixed-precision neural processing unit (NPU) using spatial fusion with load balancing

An apparatus, comprising:

• a machine-learning system comprising:
• a first computation circuit configured to compute a first partial product based on a first multiplicand and a second multiplicand loaded into the first computation circuit;
• a second computation circuit configured to compute a second partial product based on a third multiplicand and a fourth multiplicand loaded into the second computation circuit;
• a precision-determination circuit configured to separate a first value of a data set into a second value and a third value, the first value comprising a first predetermined number of bits, the second value comprising a second predetermined number of contiguous most-significant bits of the first value and the third value comprising a third predetermined number of contiguous least-significant bits of the first value, the first predetermined number equaling the second predetermined number plus the third predetermined number;
• a load-balancing circuit configured to:
• select a first selection and a second selection from:
• a first combination comprising the second value and the second predetermined number of contiguous bits of a first weight value corresponding to the first value, the second value comprising a first multiplicand of the first combination and the second predetermined number of contiguous bits of the first weight value comprising a second multiplicand of the first combination, and
• a second combination comprising the third value and the third predetermined number of bits of the first weight value, the third value comprising a first multiplicand of the second combination and the third predetermined number of contiguous bits of the first weight value comprising a second multiplicand of the second combination,
• load the first computation circuit with the first selection based on the load-balancing circuit being configured to alternate between loading the first computation circuit with (a) most-significant bits of a first data set and most-significant bits of a weight value of the first data set, and (b) least-significant bits of a second data set and least-significant bits of a weight value of the second data set, the first multiplicand of the first selection being the first multiplicand loaded in the first computation circuit and the second multiplicand of the first selection being the second multiplicand loaded in the first computation circuit, and
• load the second computation circuit with the second selection based on the load-balancing circuit being configured to alternate between loading the second computation circuit with (a) least-significant bits of the first data set and least-significant bits of the weight value of the first data set, and (b) most-significant bits of the second data set and most-significant bits of the weight value of the second data set, the first multiplicand of the second selection being the third multiplicand loaded in the second computation circuit and the second multiplicand of the second selection being the fourth multiplicand loaded in the second computation circuit;
• and
• a fusion circuit configured to combine the first partial product with the second partial product to form a combined computation result.

An apparatus configured to provide a neural network, the apparatus comprising:

• a first computation circuit configured to compute a first partial product associated with a first value of a first cell of data, the first partial product comprising a product of a first data subdivision of the first value and a first weight value corresponding to the first value, each first value comprising a first predetermined number of bits and each first data subdivision of the first value comprising a second predetermined number of contiguous most-significant bits of the first value, the second predetermined number being less than the first predetermined number;
• a second computation circuit configured to compute a second partial product associated with the first value, the second partial product comprising a product of a second data subdivision of the first value and the first weight value, the second partial product being computed concurrently with computation of the first partial product, the second data subdivision of the first value comprising a third predetermined number of contiguous least-significant bits of the first value, the second predetermined number plus the third predetermined number equaling the first predetermined number;
• a load-balancing circuit configured to:
• select a load-balancing technique that includes loading combinations of first data subdivisions and first weight values corresponding to the first data subdivisions, and loading combinations of second data subdivisions and first weight values corresponding to the second data subdivisions between the first computation circuit and the second computation circuit to reduce desynchronization between the partial products of the first computation circuit and the second computation circuit,
• alternate between loading the first computation circuit with (a) most-significant bits of a first data set and most-significant bits of a weight value of the first data set, and (b) least-significant bits of a second data set and least-significant bits of a weight value of the second data set;
• and
• alternate between loading the second computation circuit with (a) least-significant bits of the first data set and least-significant bits of the weight value of the first data set, and (b) most-significant bits of the second data set and most-significant bits of the weight value of the second data set;
• and
• a fusion circuit configured to combine the first partial product with the second partial product to form a combined computation result.