Neural network instruction set architecture

An electronic system comprising multiple hardware compute units for accelerating inference computations for a neural network having a plurality of neural network layers, the electronic system comprising:

• at least one processing unit, the at least one processing unit including one or more processing devices;
• and
• one or more non-transitory machine-readable storage devices for storing instructions that are executable by the one or more processing devices to cause performance of operations comprising:
• providing, by a controller of the electronic system, a respective single instruction to each hardware compute unit of the multiple hardware compute units, wherein each single instruction provided to the hardware compute units:
• encodes a neural network layer type of a particular neural network layer of the plurality of neural network layers and identifies data values for performing a tensor computation for the particular neural network layer;
• receiving, by a respective processing unit in each of the hardware compute units, the respective single instruction provided by the controller of the electronic system;
• generating, by the respective processing unit in each of the hardware compute units, a respective loop nest comprising a plurality of loops, wherein the loop nest is generated based on the neural network layer type encoded by the single instruction;
• and
• performing, by the respective processing unit in each of the hardware compute units, a respective portion of the tensor computation by executing the loop nest comprising the plurality of loops generated by the processing unit, wherein the respective portion of the tensor computation performed by each of the hardware compute units is at least a subset of computations for a respective partition of the particular neural network layer.

A non-transitory computer-readable storage medium comprising instructions for accelerating inference computations for a neural network having a plurality of neural network layers and using a system comprising multiple hardware compute units, the instructions being executable by at least one processing unit which, upon such execution, causes performance of operations comprising:

• providing, by a controller of the system, a respective single instruction to each hardware compute unit of the multiple hardware compute units, wherein each single instruction provided to the hardware compute units:
• encodes a neural network layer type of a particular neural network layer of the plurality of neural network layers and identifies specific data values for performing a tensor computation for the particular neural network layer;
• receiving, by a processing unit in each of the hardware compute units, the respective single instruction provided by the controller of the system;
• generating, by the respective processing unit in each of the hardware compute units, a respective loop nest comprising a plurality of loops, wherein the loop nest is generated based on the neural network layer type encoded by the single instruction;
• and
• performing, by the respective processing unit in each of the hardware compute units, a respective portion of the tensor computation by executing the loop nest comprising the plurality of loops generated by the processing unit, wherein the respective portion of the tensor computation performed by each of the hardware compute units is at least a subset of computations for a respective partition of the particular neural network layer.