# US 12,596,928 B2 · Movement of tensor data during reshape operation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,596,928 B2 - Document ID: US12596928B2 - Status: Granted - Kind: B2 - Application number: 17/792,117 - Filed: 2020-03-30 - Published: 2023-02-16 - Granted: 2026-04-07 - Earliest claimed: 2020-03-30 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Arun Chauhan; Fatih Mehmet Bakir; Phitchaya Mangpo Phothilimthana; Dong Hyuk Woo - Web page: https://patentgrep.com/patent/US12596928B2 ## CPC - G06N 3/063: using electronic means - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals} - G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method of performing a reshape operation specified in a reshape layer of a neural network model is described. The reshape operation reshapes an input tensor with an input tensor shape to an output tensor with an output tensor shape. The tensor data that has to be reshaped is directly routed between tile memories of the hardware accelerator in an efficient manner. This advantageously optimizes usage of memory space and allows any number and type of neural network models to be run on the hardware accelerator. ## Claims ### Claim 1 A method of performing a reshape operation specified in a reshape layer of a neural network model, the method comprising: transmitting, to a reshape solver, an input tensor with an input tensor shape and an output tensor with an output tensor shape; receiving, from the reshape solver, data identifying chunks of tensor data to be moved within memories of a hardware accelerator, a source computing unit on the hardware accelerator from where each corresponding chunk is to be moved, and a target computing unit on the hardware accelerator to where each corresponding chunk is to be moved; transmitting, to a constraint based solver, the received data and a maximum number of time steps over which the reshape operation is to be performed; receiving, from the constraint based solver, a schedule based on the received data, a number of computing units within the hardware accelerator, and a maximum number of time steps, the schedule indicating routes for transferring the chunks of tensor data between the memories of the hardware accelerator; updating the schedule by removing cyclical routes or merging chunks moving over a single route or both to generate an updated schedule; compiling the updated schedule to generate compiled data; and transmitting the compiled data to the hardware accelerator. ### Claim 2 (depends on 1) The method of claim 1, further comprising: creating, by the hardware accelerator, buffers in response to the compiled data to temporarily store corresponding chunks being moved within memories of the hardware accelerator. ### Claim 3 (depends on 1) The method of claim 1, wherein the reshape solver is programmed with a first set of one or more constraints. ### Claim 4 (depends on 1) The method of claim 1, wherein: the transmitting to the reshape solver, the receiving from the reshape solver, the transmitting to the constraint based solver, and the receiving from the constraint based solver are performed by a central processing unit (CPU) that implements a compiler; and the compiling of the schedule and the transmitting to the hardware accelerator are performed by the compiler. ### Claim 5 (depends on 1) The method of claim 1, further comprising: updating the received data to remove one or more chunks for which the source computing unit and the target computing unit are adjacently arranged within the hardware accelerator, the updating being performed subsequent to the receiving from the reshape solver of the received data and prior to the transmitting to the constraint based solver of the received data. ### Claim 6 (depends on 2) The method of claim 2, wherein a storage capacity of each buffer depends on the reshape operation. ### Claim 7 (depends on 3) The method of claim 3, wherein the constraint based solver is programmed with a second set of one or more constraints. ### Claim 8 A system that performs a reshape operation specified in a reshape layer of a neural network model, the reshape operation configured to reshape an input tensor with an input tensor shape to an output tensor with an output tensor shape, the system comprising: at least one programmable processor; and a machine-readable medium storing instructions that, when executed by the at least one programmable processor, cause the at least one programmable processor to: transmit, to a reshape solver, the input tensor and the output tensor; receive, from the reshape solver, data identifying chunks of tensor data to be moved within memories of a hardware accelerator, a source computing unit on the hardware accelerator from where each corresponding chunk is to be moved, and a target computing unit on the hardware accelerator to where each corresponding chunk is to be moved; transmit, to a constraint based solver, the received data, a number of computing units within the hardware accelerator, and a maximum number of time steps over which the reshape operation is to be performed; receive, from the constraint based solver, a schedule based on the received data, the number of computing units within the hardware accelerator, and the maximum number of time steps, the schedule indicating routes for transferring the chunks of tensor data between the memories of the hardware accelerator; updating the schedule by removing cyclical routes or merging chunks moving over a single route or both to generate an updated schedule; compile the updated schedule to generate compiled data; and transmit the compiled data to the hardware accelerator. ### Claim 9 (depends on 8) The system of claim 8, wherein the hardware accelerator is configured to create buffers in response to the compiled data to temporarily store corresponding chunks being moved within memories of the hardware accelerator. ### Claim 10 (depends on 8) The system of claim 8, wherein the reshape solver is programmed with a first set of one or more constraints. ### Claim 11 (depends on 8) The system of claim 8, wherein: the at least one programmable processor is a central processing unit (CPU) that implements a compiler; and the compiling of the schedule and the transmitting of the compiled data to the hardware accelerator are performed by the compiler. ### Claim 12 (depends on 8) The system of claim 8, wherein the at least one programmable processor is configured to update the received data to remove one or more chunks for which the source computing unit and the target computing unit are adjacently arranged within the hardware accelerator, the updating being performed subsequent to the receiving from the reshape solver of the received data and prior to the transmitting to the constraint based solver of the received data. ### Claim 13 (depends on 9) The system of claim 9, wherein a storage capacity of each buffer depends on the reshape operation. ### Claim 14 (depends on 10) The system of claim 10, wherein the constraint based solver is programmed with a second set of one or more constraints. ### Claim 15 A non-transitory computer program product storing instructions that, when executed by at least one programmable processor, cause the at least one programmable processor to: transmit, to a reshape solver, an input tensor with an input tensor shape and an output tensor with an output tensor shape; receive, from the reshape solver, data identifying chunks of tensor data to be moved within memories of a hardware accelerator, a source computing unit on the hardware accelerator from where each corresponding chunk is to be moved, and a target computing unit on the hardware accelerator to where each corresponding chunk is to be moved; transmit, to a constraint based solver, the received data, a number of computing units within the hardware accelerator, and a maximum number of time steps over which a reshape operation is to be performed; receive, from the constraint based solver, a schedule based on the received data, the number of computing units within the hardware accelerator, and the maximum number of time steps, the schedule indicating routes for transferring the chunks of tensor data between the memories of the hardware accelerator; updating the schedule by removing cyclical routes or merging chunks moving over a single route or both to generate an updated schedule; compile the updated schedule to generate compiled data; and transmit the compiled data to the hardware accelerator. ### Claim 16 (depends on 15) The non-transitory computer program product of claim 15, wherein the at least one programmable processor is configured to update the received data to remove one or more chunks for which the source computing unit and the target computing unit are adjacently arranged within the hardware accelerator, the updating being performed subsequent to the receiving from the reshape solver of the received data and prior to the transmitting to the constraint based solver of the received data.