Methods and systems for handling operations in sequence for neural network processing

A processor comprising one or more functional units operable to execute processing for neural network processing, the processor comprising:

• a local storage operable to store data for neural network processing when the one or more functional units of the processor are performing processing for neural network processing;
• and
• a control circuit configured to:
• cause the one or more functional units of the processor to perform processing for neural network processing, comprising a sequence of operations to process an initial input data array for the sequence of operations to generate a final output data array of the sequence of operations for respective blocks of the initial input data array on a block-by-block basis, such that the initial input data array is processed as plural blocks which are each processed independently according to the sequence of operations to generate a corresponding block of the final output data array;
• wherein:
• at least one of the operations in the sequence of operations uses output data from a previous operation in the sequence of operations as input data;
• and to:
• cause the one or more functional units of the processor to perform processing for the neural network processing, comprising the sequence of operations for a block of an initial input data array by:
• performing the operations of the sequence of operations one after another for the block of the initial input data array to generate a block of the final output data array;
• and:
• when performing an operation in the sequence of operations whose output data is to be used as input data for another operation in the sequence of operations, storing the output data which is to be used as input data for another operation of the sequence of operations in the local storage, and retrieving the output data for use as input data for the another operation in the sequence of operations from the local storage;
• and:
• for a final operation in the sequence of operations, storing output data generated by the final operation corresponding to a block of the final output data array in a main storage.

A non-transitory computer readable storage medium storing computer software code which when executing on a processor performs a method of compiling a neural network to be executed by a processor operable to perform neural network processing, the method comprising:

• for a neural network comprising a set of plural neural network processing operations to be performed:
• determining whether there are any sequences of operations in the set of operations for the neural network that can be performed by one or more functional units of the processor as a chained sequence of operations in which:
• the processing of an initial input data array for the sequence of operations through the sequence of operations to generate a final output data array of the sequence of operations can be performed on a block-by-block basis, such that the initial input data array is processed as one or more blocks which are each processed independently according to the sequence of operations to generate a corresponding block of the final output data array for the sequence of operations;
• and
• at least one of the operations in the sequence of operations uses output data from a previous operation in the sequence of operations as input data;
• the method further comprising, when it is determined that there is a sequence of operations in the set of operations for the neural network that can be performed by one or more functional units of a processor as a chained sequence of operations:
• generating commands and/or descriptors that when executed by the one or more functional units of the processor will cause the one or more functional units to:
• perform processing for the neural network processing comprising the determined sequence of operations to process an initial input data array for the sequence of operations to generate a final output data array of the sequence of operations for respective blocks of the initial input data array on a block-by-block basis, such that the initial input data array is processed as plural blocks which are each processed independently according to the sequence of operations to generate a corresponding block of the final output data array;
• and
• perform processing for the neural network processing comprising the sequence of operations for a block of the initial input data array by:
• performing the operations of the sequence of operations one after another for the block of the initial input data array to generate a block of the final output data array;
• and
• when performing an operation in the sequence of operations whose output data is to be used as input data for another operation in the sequence of operations, storing the output data which is to be used as input data for another operation of the sequence of operations in local storage for the one or more functional units of the processor that are performing the processing for the neural network processing, and providing the output data as input data for the another operation in the sequence of operations from the local storage;
• and
• for a final operation in the sequence of operations, storing output data generated by the final operation corresponding to a block of the final output data array in a main storage.