Neural network instruction set architecture

A system for accelerating tensor computations for a neural network implemented on a hardware integrated circuit and having a plurality of neural network layers, the system comprising:

• a plurality of compute units that are individual compute tiles of the hardware integrated circuit;
• a controller configured to:
• i) obtain a single instruction encoding at least one opcode that specifies a layer type of a neural network layer and ii) provide the single instruction to each of the plurality of compute units,
• wherein, for each of the plurality of compute units:
• the single instruction, when executed by a respective processor of the compute unit, causes the compute unit to perform operations comprising:
• determining that the tensor computations are for a particular type of neural network layer based on an opcode in the single instruction that specifies the layer type of the neural network layer from a plurality of neural network layer types that includes:
• a convolutional layer type and a fully connected layer type;
• determining, based on the single instruction and the layer type of the neural network layer, a compute sequence for executing a respective set of operations to process data values at a portion of an input tensor through the neural network layer;
• and
• performing, based on the compute sequence and the single instruction, the tensor computations for the particular type of neural network layer using inputs and weights obtained from memory address locations accessed at the compute unit, comprising:
• traversing, based on the single instruction, elements along one or more dimensions of the input tensor using a respective hardware processing unit of the compute unit;
• and
• generating, based on the single instruction, an output value for the particular type of neural network layer in response to performing the tensor computations at the compute unit.

One or more non-transitory machine-readable storage devices for storing instructions used to accelerate tensor computations for a neural network implemented on a hardware integrated circuit and having a plurality of neural network layers, the instructions being executable by one or more processors to cause performance of operations comprising:

• obtaining, using a controller of the hardware integrated circuit, a single instruction encoding at least one opcode that specifies a layer type of a neural network layer;
• providing, by the controller, the single instruction to each of a plurality of compute units that are individual compute tiles of the hardware integrated circuit;
• for each of the plurality of compute units:
• the single instruction, when executed by a respective processor of the compute unit, causes the compute unit to perform operations comprising:
• determining that the tensor computations are for a particular type of neural network layer based on an opcode in the single instruction that specifies the layer type of the neural network layer from a plurality of neural network layer types that includes:
• a convolutional layer type and a fully connected layer type;
• determining, based on the single instruction and the layer type of the neural network layer, a compute sequence for executing a respective set of operations to process data values at a portion of an input tensor through the neural network layer;
• and
• performing, based on the compute sequence and the single instruction, the tensor computations for the particular type of neural network layer using inputs and weights obtained from memory address locations accessed at the compute unit, comprising:
• traversing, based on the single instruction, elements along one or more dimensions of the input tensor using a responsive hardware processing unit of the compute unit;
• and
• generating, based on the single instruction, an output for the particular type of neural network layer in response to performing the tensor computations across the plurality of compute units.