# US 12,762,385 B2 · Reduction operation with retention in neural network processor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,762,385 B2 - Document ID: US12762385B2 - Status: Granted - Kind: B2 - Application number: 17/505,426 - Filed: 2021-10-19 - Published: 2023-04-20 - Granted: 2026-10-13 - Earliest claimed: 2021-10-19 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Christopher L. Mills; Youchang Kim - Web page: https://patentgrep.com/patent/US12762385B2 ## CPC - G06N 3/063: using electronic means - G06F 17/153: {Multidimensional correlation or convolution} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Embodiments of the present disclosure relate to a reduction operation in a neural processor circuit where results of the reduction operation are retained for multiple post-processing operations. The neural processor circuit includes neural engine circuits and a planar engine circuit coupled to the neural engine circuits. At least one neural engine circuit performs a convolution operation to generate output data. The planar engine circuit includes a filter circuit and a line buffer coupled to the filter circuit. The filter circuit performs a reduction operation for each patch of a tensor from the output data to generate a respective reduced value associated with a corresponding channel of the tensor. The line buffer stores reduced values each being associated with a respective channel of the tensor. The line buffer retains the reduced values for a defined number of operating cycles as indicated by a refresh flag defining resetting of the line buffer. ## Claims ### Claim 1 A neural processor circuit, comprising: a plurality of neural engine circuits comprising a first neural engine circuit and a second neural engine circuit, at least one of the plurality of neural engine circuits configured to perform a convolution operation to generate output data; a planar engine circuit that is separated from and coupled to the plurality of neural engine circuits including the first neural engine circuit and the second neural engine circuit, the planar engine circuit including: a first filter circuit configured to perform a reduction operation to reduce a patch tensor of a first rank to another tensor of a second rank lower than the first rank by removing one or more dimensions of the patch tensor of the first rank for each patch of a plurality of patches of a tensor for a dataset of an input data originating from the output data to generate a respective reduced value of a plurality of reduced values, the respective reduced value associated with a corresponding channel of a plurality of channels of the tensor for the dataset, the respective reduced value being generated based on values of elementwise operations between a first work unit of each patch and a second work unit of another patch, wherein the tensor of the input data comprises more than 3 dimensions and the plurality of patches, at least one patch of the plurality of patches includes 3 dimensions and a plurality of work units, and at least one work unit of the plurality of work units represents the dataset of a size capable of being processed by the planar engine circuit in a single operating cycle; a second filter circuit coupled to the first filter circuit, the second filter circuit configured to: receive a first tensor and a second tensor as part of the output data; and perform an elementwise operation between the first tensor and the second tensor to generate the tensor for the first filter circuit; and a line buffer coupled to the first filter circuit, the line buffer configured to: store the plurality of reduced values for the plurality of channels being transferred from one or more registers within the first filter circuit, and retain the plurality of reduced values for a defined number of operating cycles as indicated by a refresh flag stored in another register external to the first filter circuit and defining resetting of the line buffer; and a data processor circuit coupled to the planar engine circuit and the plurality of neural engine circuits, the data processor circuit configured to: store the output data obtained from at least one neural engine circuit; and send the output data to the planar engine circuit, the output data comprising the dataset as the input data for the plurality of patches and the plurality of channels of the tensor. ### Claim 2 (depends on 1) The neural processor circuit of claim 1, wherein the planar engine circuit further comprises a post-processor circuit coupled to the line buffer, the post-processor circuit configured, during the defined number of operating cycles, to: perform a first post-processing operation on a channel vector comprising the plurality of reduced values from the line buffer to generate a first output vector; and perform a second post-processing operation on the channel vector following the first post-processing operation to generate a second output vector. ### Claim 3 (depends on 1) The neural processor circuit of claim 1, wherein each patch comprises a first number of work units, the respective reduced value being stored in the one or more registers within the first filter circuit configured to store the plurality of reduced values generated in a second number of work units before the respective reduced value being transferred to the line buffer, wherein the first number is same as the second number. ### Claim 4 (depends on 1) The neural processor circuit of claim 1, wherein: the first filter circuit is further configured to determine indexing information for each reduced value of the plurality of reduced values; and the line buffer is further configured to: store the indexing information for each reduced value, and retain the indexing information for the defined number of operating cycles as indicated by the refresh flag. ### Claim 5 (depends on 4) The neural processor circuit of claim 4, wherein the indexing information for each reduced value comprises information about spatial coordinates of that reduced value in the tensor. ### Claim 6 (depends on 1) The neural processor circuit of claim 1, wherein the first filter circuit is further configured to: perform the reduction operation by finding a respective minimum value or a respective maximum value for a corresponding channel of the plurality of channels of the tensor for the dataset; and determine corresponding indexing information for the respective minimum value or the respective maximum value. ### Claim 7 (depends on 6) The neural processor circuit of claim 6, wherein the line buffer is further configured to: store the respective minimum value or the respective maximum value and the corresponding indexing information; and retain the respective minimum value or the respective maximum value and the corresponding indexing information for the defined number of operating cycles as indicated by the refresh flag. ### Claim 8 (depends on 1) The neural processor circuit of claim 1, wherein: a first value of the refresh flag initiates an initialization of the line buffer prior to the reduction operation; and a second value of the refresh flag configures the line buffer to skip the initialization and retain the plurality of reduced values for a plurality of post-processing operations performed after the reduction operation. ### Claim 9 (depends on 1) The neural processor circuit of claim 1, wherein the first filter circuit is further configured to: perform, during the defined number of operating cycles, at least one reduction operation following the reduction operation, the at least one reduction operation comprises operations not affecting the plurality of reduced values retained in the line buffer for the defined number of operating cycles. ### Claim 10 A method of operating a neural processor circuit, comprising: operating at least one of a plurality of neural engine circuits comprising a first neural engine circuit and a second neural engine circuit in the neural processor circuit to perform a convolution operation to generate output data; performing, at a first filter circuit of a planar engine circuit in the neural processor circuit that is separated from and coupled to the plurality of neural engine circuits comprising the first neural engine circuit and the second neural engine circuit, a reduction operation to reduce a patch tensor of a first rank to another tensor of a second rank lower than the first rank by removing one or more dimensions of the patch tensor of the first rank for each patch of a plurality of patches of a tensor for a dataset of an input data originating from the output data to generate a respective reduced value of a plurality of reduced values, the respective reduced value associated with a corresponding channel of a plurality of channels of the tensor for the dataset, the respective reduced value being generated based on values of elementwise operations between a first work unit of each patch and a second work unit of another patch, wherein the tensor of the input data comprises more than 3 dimensions and the plurality of patches, at least one patch of the plurality of patches includes 3 dimensions and a plurality of work units, and at least one work unit of the plurality of work units represents the dataset of a size capable of being processed by the planar engine circuit in a single operating cycle; storing, in a line buffer of the planar engine circuit coupled to the first filter circuit, the plurality of reduced values for the plurality of channels being transferred from one or more registers within the first filter circuit; receiving, at a second filter circuit of the planar engine circuit coupled to the first filter circuit, a first tensor and a second tensor as part of the output data; performing, at the second filter circuit, an elementwise operation between the first tensor and the second tensor to generate the tensor for the first filter circuit; retaining, in the line buffer, the plurality of reduced values for a defined number of operating cycles as indicated by a refresh flag stored in another register external to the first filter circuit and defining resetting of the line buffer; storing, at a data processor circuit coupled to the planar engine circuit and the plurality of neural engine circuits, the output data obtained from at least one neural engine circuit; and sending the output data from the data processor circuit to the planar engine circuit, the output data comprising the dataset as the input data for the plurality of patches and the plurality of channels of the tensor. ### Claim 11 (depends on 10) The method of claim 10, further comprising: performing, at a post-processor circuit of the planar engine circuit coupled to the line buffer, a first post-processing operation on a channel vector comprising the plurality of reduced values from the line buffer to generate a first output vector; and performing, at the post-processor circuit, a second post-processing operation on the channel vector following the first post-processing operation to generate a second output vector. ### Claim 12 (depends on 10) The method of claim 10, wherein each patch comprises a first number of work units, the respective reduced value being stored in the one or more registers within the first filter circuit configured to store the plurality of reduced values generated in a second number of work units before the respective reduced value being transferred to the line buffer, wherein the first number is same as the second number. ### Claim 13 (depends on 10) The method of claim 10, further comprising: determining, at the first filter circuit, indexing information for each reduced value that comprises information about spatial coordinates of that reduced value in the tensor; storing, in the line buffer, the indexing information for each reduced value; and retaining, in the line buffer, the indexing information for the defined number of operating cycles as indicated by the refresh flag. ### Claim 14 (depends on 10) The method of claim 10, further comprising: performing the reduction operation by finding a respective minimum value or a respective maximum value for a corresponding channel of the plurality of channels of the tensor; and determining corresponding indexing information for the respective minimum value or the respective maximum value. ### Claim 15 (depends on 14) The method of claim 14, further comprising: storing, in the line buffer, the respective minimum value or the respective maximum value and the corresponding indexing information; and retaining the respective minimum value or the respective maximum value and the corresponding indexing information in the line buffer for the defined number of operating cycles as indicated by the refresh flag. ### Claim 16 (depends on 10) The method of claim 10, further comprising: initiating an initialization of the line buffer prior to the reduction operation using a first value of the refresh flag; configuring, using a second value of the refresh flag, the line buffer to skip the initialization and retain the plurality of reduced values for a plurality of post-processing operations performed after the reduction operation; and performing, at the first filter circuit during the defined number of operating cycles, at least one reduction operation following the reduction operation, the at least one reduction operation comprises operations not affecting the plurality of reduced values retained in the line buffer for the defined number of operating cycles. ### Claim 17 An electronic device, comprising: a system memory storing input data; and a neural processor circuit coupled to the system memory, the neural processor circuit including: a data processor circuit configured to receive the input data from the system memory, a plurality of neural engine circuits comprising a first neural engine circuit and a second neural engine circuit coupled to the data processor circuit, at least one of the plurality of neural engine circuits configured to perform a convolution operation on the input data to generate output data for storage into the data processor circuit, and a planar engine circuit that is separated from and coupled to the plurality of neural engine circuits comprising the first neural engine circuit and the second neural engine circuit, and further coupled to the data processor circuit, the planar engine circuit including: a first filter circuit configured to perform a reduction operation to reduce a patch tensor of a first rank to another tensor of a second rank lower than the first rank by removing one or more dimensions of the patch tensor of the first rank for each patch of a plurality of patches of a tensor for a dataset of the input data originating from the output data to generate a respective reduced value of a plurality of reduced values, the respective reduced value being associated with a corresponding channel of a plurality of channels of the tensor for the dataset, the respective reduced value being generated based on values of elementwise operations between a first work unit of each patch and a second work unit of another patch, wherein the tensor of the input data comprises more than 3 dimensions and the plurality of patches, at least one patch of the plurality of patches includes 3 dimensions and a plurality of work units, and at least one work unit of the plurality of work units represents the dataset of a size capable of being processed by the planar engine circuit in a single operating cycle; a second filter circuit coupled to the first filter circuit, the second filter circuit configured to: receive a first tensor and a second tensor as part of the output data; and perform an elementwise operation between the first tensor and the second tensor to generate the tensor for the first filter circuit; and a line buffer coupled to the first filter circuit, the line buffer configured to: store the plurality of reduced values for the plurality of channels being transferred from one or more registers within the first filter circuit, and retain the plurality of reduced values for a defined number of operating cycles as indicated by a refresh flag stored in another register external to the first filter circuit and defining resetting of the line buffer; and wherein the data processor circuit is configured to: store the output data obtained from at least one neural engine circuit; and send the output data to the planar engine circuit, the output data comprising the dataset as the input data for the plurality of patches and the plurality of channels of the tensor. ### Claim 18 (depends on 17) The electronic device of claim 17, wherein the planar engine circuit further comprises a post-processor circuit coupled to the line buffer and the data processor circuit, the post-processor circuit configured, during the defined number of operating cycles, to: perform a first post-processing operation on a channel vector comprising the plurality of reduced values from the line buffer to generate a first output vector for storage into the data processor circuit; and perform a second post-processing operation on the channel vector following the first post-processing operation to generate a second output vector for storage into the data processor circuit. ### Claim 19 (depends on 17) The electronic device of claim 17, wherein each patch comprises a first number of work units, the respective reduced value being stored in the one or more registers within the first filter circuit configured to store the plurality of reduced values generated in a second number of work units before the respective reduced value being transferred to the line buffer, wherein the first number is same as the second number. ### Claim 20 (depends on 17) The electronic device of claim 17, wherein: the first filter circuit is further configured to determine indexing information for each reduced value of the plurality of reduced values; and the line buffer is further configured to: store the indexing information for each reduced value, and retain the indexing information for the defined number of operating cycles as indicated by the refresh flag.