# US 12,602,185 B2 · Maintaining synchronisation between memory writing and reading blocks using an internal buffer and a control channel > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,602,185 B2 - Document ID: US12602185B2 - Status: Granted - Kind: B2 - Application number: 18/376,094 - Filed: 2023-10-03 - Published: 2024-01-25 - Granted: 2026-04-14 - Earliest claimed: 2020-06-02 - Assignees: Imagination Technologies Limited - Applicants: Imagination Technologies Limited - Inventors: Alan Vines; Stephen Spain; Fernando Escobar - Web page: https://patentgrep.com/patent/US12602185B2 ## CPC - G06F 3/0656: {Data buffering arrangements} - G06F 13/1673: {using buffers} - G06F 3/061: {Improving I/O performance} - G06F 3/0679: {Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]} - G06F 13/1689: {Synchronisation and timing concerns (synchronisation on a memory bus G06F13/4234)} - G06N 3/063: using electronic means - Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract A hardware unit for manipulating data stored in a memory comprises an internal buffer, a memory reading block, configured to read the data from the memory and write the data to the internal buffer. a memory writing block, configured to read the data from the internal buffer and write the data to the memory. The hardware unit optionally also comprises a control channel between the memory reading block and the memory writing block, wherein the memory reading block and the memory writing block are configured to communicate via the control channel to maintain synchronisation between them when writing the data to the internal buffer and reading the data from the internal buffer, respectively. The hardware unit may be configured to apply one or more transformations to multidimensional data in the memory. The hardware unit may be configured to traverse the multidimensional array using a plurality of nested loops. ## Claims ### Claim 1 A hardware unit for manipulating data stored in a memory, the hardware unit comprising: an internal buffer; a memory reading block, configured to read the data from the memory and write the data to the internal buffer; a memory writing block, configured to read the data from the internal buffer and write the data to the memory; and a control channel between the memory reading block and the memory writing block, wherein the memory reading block and the memory writing block are configured to communicate via the control channel to maintain synchronisation between them when writing the data to the internal buffer and reading the data from the internal buffer, respectively; wherein the data comprises a multidimensional array comprising a plurality of data elements, wherein at least one of the memory reading block and the memory writing block is configured to traverse the multidimensional array using a plurality of nested loops, each loop having associated with it a corresponding stride between data elements of the multidimensional array. ### Claim 2 (depends on 1) The hardware unit of claim 1, wherein at least one loop of the plurality of nested loops is configured to iterate a different number of times depending on at least one of: (a) a loop index of at least one other loop of the plurality of nested loops; and (b) a software configurable flag. ### Claim 3 (depends on 1) The hardware unit of claim 1, wherein each loop of the plurality of loops is configured to perform a variable number of iterations, the variable number being selected at runtime from a group comprising: a first number of iterations to be performed when one or more outer loops of the plurality of nested loops are not in their end iteration; and a second number of iterations to be performed when the one or more outer loops of the plurality of nested loops are in their end iteration. ### Claim 4 (depends on 1) The hardware unit of claim 1, wherein the multidimensional array is stored in the memory in a storage format having storage units of a predetermined third size, wherein one or more dimensions of the multidimensional array are not an integer multiple of the third size. ### Claim 5 (depends on 4) The hardware unit of claim 4, wherein at least one of the memory reading block and the memory writing block is configured to traverse the multidimensional array using a plurality of nested loops, each loop having associated with it a corresponding stride between data elements of the multidimensional array, wherein said at least one block is configured to select, for at least one loop of the plurality of nested loops, a different number of iterations when one or more outer loops are in their end iteration, as compared with the number of iterations of said at least one loop when the one or more outer loops are not in their end iteration. ### Claim 6 (depends on 1) The hardware unit of claim 1, wherein the synchronisation between the memory reading block and memory writing block ensures that no data is overwritten in the internal buffer by the memory reading block while it is still needed by the memory writing block, and that data is not read from the internal buffer by the memory writing block until it has been validly written to the internal buffer by the memory reading block. ### Claim 7 A hardware unit for manipulating data stored in a memory, the hardware unit comprising: an internal buffer; a memory reading block, configured to read the data from the memory and write the data to the internal buffer; a memory writing block, configured to read the data from the internal buffer and write the data to the memory; and a control channel between the memory reading block and the memory writing block, wherein the memory reading block and the memory writing block are configured to communicate via the control channel to maintain synchronisation between them when writing the data to the internal buffer and reading the data from the internal buffer, respectively, wherein each of the memory reading block and the memory writing block has a respective synchronisation counter, the blocks being configured to communicate their synchronisation counters with each other via the control channel, wherein the hardware unit is configured to maintain synchronisation between the blocks by comparing the synchronisation counters. ### Claim 8 (depends on 7) The hardware unit of claim 7, wherein the data comprises a multidimensional array comprising a plurality of data elements, wherein each of the memory reading block and the memory writing block is configured to traverse the multidimensional array using a respective plurality of nested loops, each loop having associated with it a corresponding stride between data elements of the multidimensional array, wherein the synchronisation counter of the memory reading block is associated with a first loop of its plurality of loops and the synchronisation counter of the memory writing block is associated with a second loop of its plurality of loops. ### Claim 9 (depends on 7) The hardware unit of claim 7, wherein the operation of one of the blocks is paused if the difference between the respective synchronisation counters exceeds a predetermined threshold. ### Claim 10 A hardware-implemented method of manipulating data stored in a memory, the data comprising a multidimensional array comprising a plurality of data elements, the method comprising: (i) reading the data from the memory and writing the data to the internal buffer using a memory reading block; and (ii) reading the data from the internal buffer and writing the data to the memory using a memory writing block, wherein at least one of the steps (i) and (ii) is performed using a plurality of nested loops, each loop having associated with it a corresponding stride between data elements of the multidimensional array, wherein at least one loop of the plurality of nested loops is configured to iterate a different number of times depending on a loop index of at least one other loop of the plurality of nested loops, and wherein the memory reading block and the memory writing block are configured to communicate via a control channel to maintain synchronisation between step (i) and step (ii); wherein each loop of the plurality of loops is configured to perform a variable number of iterations, the variable number being selected at runtime from a group comprising: a first number of iterations to be performed when one or more outer loops of the plurality of nested loops are in their first iteration; and a second number of iterations to be performed when the one or more outer loops of the plurality of nested loops are in their end iteration. ### Claim 11 (depends on 10) The method of claim 10, wherein at least one loop of the plurality of nested loops is further configured to iterate a different number of times depending on a software configurable flag. ### Claim 12 (depends on 10) The method of claim 10, further comprising: maintaining a first synchronisation counter associated with a first loop among the plurality of loops performing step (i), wherein the first synchronisation counter is incremented with each iteration of the first loop; maintaining a second synchronisation counter associated with a second loop among the plurality of loops performing step (ii), wherein the second synchronisation counter is incremented with each iteration of the second loop; comparing a current value of the first synchronisation counter with a current value of the second synchronisation counter; and controlling the progress of step (i) and or step (ii) based on a result of the comparison. ### Claim 13 (depends on 12) The method of claim 12, wherein controlling the progress of step (i) and/or step (ii) based on a result of the comparison comprises one or more of: pausing step (i) if the first synchronisation counter exceeds the second synchronisation counter by more than a predetermined first threshold; and pausing step (ii) if the second synchronisation counter exceeds the first synchronisation counter by more than a predetermined second threshold. ### Claim 14 (depends on 1) A method of manufacturing, using an integrated circuit manufacturing system, the hardware unit as set forth in claim 1, the method comprising: processing, using a layout processing system, a computer readable dataset description of the hardware unit so as to generate a circuit layout description of an integrated circuit embodying the hardware unit or neural network accelerator system; and manufacturing, using an integrated circuit generation system, the hardware unit according to the circuit layout description. ### Claim 15 (depends on 10) A non-transitory computer readable storage medium having stored thereon computer readable code configured to cause the method as set forth in claim 10 to be performed when the code is run on at least one processor. ### Claim 16 (depends on 1) A non-transitory computer readable storage medium having stored thereon a computer readable data description of a hardware unit as set forth in claim 1 that, when processed in an integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying the hardware unit or neural network accelerator system. ### Claim 17 (depends on 1) An integrated circuit manufacturing system comprising: a non-transitory computer readable storage medium having stored thereon a computer readable dataset description of a hardware unit as set forth in claim 1; a layout processing system configured to process the computer readable description so as to generate a circuit layout description of an integrated circuit embodying the hardware unit; and an integrated circuit generation system configured to manufacture the hardware unit according to the circuit layout description.