Reduction operation with retention in neural network processor

A method of operating a neural processor circuit, comprising:

• operating at least one of a plurality of neural engine circuits comprising a first neural engine circuit and a second neural engine circuit in the neural processor circuit to perform a convolution operation to generate output data;
• performing, at a first filter circuit of a planar engine circuit in the neural processor circuit that is separated from and coupled to the plurality of neural engine circuits comprising the first neural engine circuit and the second neural engine circuit, a reduction operation to reduce a patch tensor of a first rank to another tensor of a second rank lower than the first rank by removing one or more dimensions of the patch tensor of the first rank for each patch of a plurality of patches of a tensor for a dataset of an input data originating from the output data to generate a respective reduced value of a plurality of reduced values, the respective reduced value associated with a corresponding channel of a plurality of channels of the tensor for the dataset, the respective reduced value being generated based on values of elementwise operations between a first work unit of each patch and a second work unit of another patch, wherein the tensor of the input data comprises more than 3 dimensions and the plurality of patches, at least one patch of the plurality of patches includes 3 dimensions and a plurality of work units, and at least one work unit of the plurality of work units represents the dataset of a size capable of being processed by the planar engine circuit in a single operating cycle;
• storing, in a line buffer of the planar engine circuit coupled to the first filter circuit, the plurality of reduced values for the plurality of channels being transferred from one or more registers within the first filter circuit;
• receiving, at a second filter circuit of the planar engine circuit coupled to the first filter circuit, a first tensor and a second tensor as part of the output data;
• performing, at the second filter circuit, an elementwise operation between the first tensor and the second tensor to generate the tensor for the first filter circuit;
• retaining, in the line buffer, the plurality of reduced values for a defined number of operating cycles as indicated by a refresh flag stored in another register external to the first filter circuit and defining resetting of the line buffer;
• storing, at a data processor circuit coupled to the planar engine circuit and the plurality of neural engine circuits, the output data obtained from at least one neural engine circuit;
• and
• sending the output data from the data processor circuit to the planar engine circuit, the output data comprising the dataset as the input data for the plurality of patches and the plurality of channels of the tensor.

An electronic device, comprising:

• a system memory storing input data;
• and
• a neural processor circuit coupled to the system memory, the neural processor circuit including:
• a data processor circuit configured to receive the input data from the system memory,
• a plurality of neural engine circuits comprising a first neural engine circuit and a second neural engine circuit coupled to the data processor circuit, at least one of the plurality of neural engine circuits configured to perform a convolution operation on the input data to generate output data for storage into the data processor circuit, and
• a planar engine circuit that is separated from and coupled to the plurality of neural engine circuits comprising the first neural engine circuit and the second neural engine circuit, and further coupled to the data processor circuit, the planar engine circuit including:
• a first filter circuit configured to perform a reduction operation to reduce a patch tensor of a first rank to another tensor of a second rank lower than the first rank by removing one or more dimensions of the patch tensor of the first rank for each patch of a plurality of patches of a tensor for a dataset of the input data originating from the output data to generate a respective reduced value of a plurality of reduced values, the respective reduced value being associated with a corresponding channel of a plurality of channels of the tensor for the dataset, the respective reduced value being generated based on values of elementwise operations between a first work unit of each patch and a second work unit of another patch, wherein the tensor of the input data comprises more than 3 dimensions and the plurality of patches, at least one patch of the plurality of patches includes 3 dimensions and a plurality of work units, and at least one work unit of the plurality of work units represents the dataset of a size capable of being processed by the planar engine circuit in a single operating cycle;
• a second filter circuit coupled to the first filter circuit, the second filter circuit configured to:
• receive a first tensor and a second tensor as part of the output data;
• and
• perform an elementwise operation between the first tensor and the second tensor to generate the tensor for the first filter circuit;
• and
• a line buffer coupled to the first filter circuit, the line buffer configured to:
• store the plurality of reduced values for the plurality of channels being transferred from one or more registers within the first filter circuit, and
• retain the plurality of reduced values for a defined number of operating cycles as indicated by a refresh flag stored in another register external to the first filter circuit and defining resetting of the line buffer;
• and
• wherein the data processor circuit is configured to:
• store the output data obtained from at least one neural engine circuit;
• and
• send the output data to the planar engine circuit, the output data comprising the dataset as the input data for the plurality of patches and the plurality of channels of the tensor.