# US 2022/0044098 A1 · Methods and systems for running dynamic recurrent neural networks in hardware > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0044098 A1 - Document ID: US20220044098A1 - Status: Published application - Kind: A1 - Application number: 17/368,384 - Filed: 2021-07-06 - Published: 2022-02-10 - Earliest claimed: 2020-07-03 - Applicants: Imagination Technologies Limited - Inventors: Muhammad Asad; Elia Condorelli; James Imber; Cagatay Dikici - Web page: https://patentgrep.com/patent/US20220044098A1 ## CPC - G06N 3/045: Combinations of networks - G06N 3/063: using electronic means - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/0495: Quantised networks; Sparse networks; Compressed networks - G06N 3/10: Interfaces, programming languages or software development kits, e.g. for simulating neural networks - G06N 3/048: Activation functions - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method of implementing in hardware a recurrent neural network (RNN) for operation on a sequence of inputs, each step of the recurrent neural network being for operation on a different input of the sequence, the method comprising: receiving a representation of the RNN; transforming the representation of the RNN into a derivative neural network for operation over a predetermined plurality of inputs of the sequence of inputs, the derivative neural network having one or more state inputs and one or more state outputs and being equivalent to the RNN over a predetermined plurality of steps of the RNN; and iteratively applying the derivative neural network to the sequence of inputs by: implementing a sequence of instances of the derivative neural network in hardware; and providing the one or more state outputs from each instance of the derivative neural network at the hardware as the one or more state inputs to a subsequent instance of the derivative neural network at the hardware so as to operate the RNN over a sequence of inputs longer than the predetermined plurality of inputs. ## Claims ### Claim 1 A method of implementing in hardware a recurrent neural network (RNN) for operation on a sequence of inputs, each step of the recurrent neural network being for operation on a different input of the sequence, the method comprising: receiving a representation of the RNN; transforming the representation of the RNN into a derivative neural network for operation over a predetermined plurality of inputs of the sequence of inputs, the derivative neural network having one or more state inputs and one or more state outputs and being equivalent to the RNN over a predetermined plurality of steps of the RNN; and iteratively applying the derivative neural network to the sequence of inputs by: implementing a sequence of instances of the derivative neural network in hardware; and providing the one or more state outputs from each instance of the derivative neural network at the hardware as the one or more state inputs to a subsequent instance of the derivative neural network at the hardware so as to operate the RNN over a sequence of inputs longer than the predetermined plurality of inputs. ### Claim 2 (depends on 1) The method of claim 1, wherein the predetermined plurality of steps is equal in number to the predetermined plurality of inputs. ### Claim 3 (depends on 1) The method of claim 1, wherein the one or more state outputs from each instance of the derivative neural network are provided as the one or more state inputs to the subsequent instance of the derivative neural network in the sequence of instances of the derivative neural network. ### Claim 4 (depends on 1) The method of claim 1, wherein the implementing a sequence of instances of the derivative neural network comprises implementing an instance of the derivative neural network and, on completion of that instance, causing the next instance of the derivative neural network in the sequence to be implemented in hardware. ### Claim 5 (depends on 1) The method of claim 1, wherein the transforming comprises unrolling the recurrent neural network over the predetermined plurality of steps so as to form the derivative neural network for operation over the predetermined plurality of inputs of the sequence of inputs. ### Claim 6 (depends on 1) The method of claim 1, wherein the hardware and its control logic are adapted to perform feed-forward neural networks. ### Claim 7 (depends on 1) The method of claim 1, wherein the hardware and its control logic are incapable of executing the received representation of the RNN. ### Claim 8 (depends on 1) The method of claim 1, wherein the hardware and its control logic are incapable of executing dynamic neural networks. ### Claim 9 (depends on 1) The method of claim 1, wherein the derivative neural network is a feed-forward neural network. ### Claim 10 (depends on 1) The method of claim 1, wherein the RNN comprises one or more cells, each cell arranged to receive a cell state input generated at a preceding step, and the transforming the representation of the RNN further comprises, at each cell: identifying non-causal operations which are for performance without dependence on the cell state input; and in the derivative neural network, grouping together at least some of the non-causal operations at a plurality of instances of the cell over at least some of the predetermined plurality of steps for processing in parallel at the hardware. ### Claim 11 (depends on 10) The method of claim 10, wherein the cell comprises causal operations which are for performance in dependence on the cell state input. ### Claim 12 (depends on 10) The method of claim 10, wherein at least part of the cell state input is generated at the preceding instance of the cell at the preceding step. ### Claim 13 (depends on 10) The method of claim 10, wherein the grouping together comprises combining the at least some non-causal operations for performance as a single convolution operation for the plurality of instances of the cell in the derivative neural network. ### Claim 14 (depends on 10) The method of claim 10, wherein the transforming the representation of the RNN further comprises splitting the at least some of the non-causal operations from the causal operations. ### Claim 15 (depends on 11) The method of claim 11, wherein the implementing a sequence of instances of the derivative neural network in hardware comprises, for each instance, causing the hardware to process one or more of the groups of non-causal operations in parallel. ### Claim 16 (depends on 11) The method of claim 11, wherein the hardware comprises an accelerator having a plurality of processing elements for executing a neural network and each group of non-causal operations is processed in parallel over the at least some of the plurality of processing elements. ### Claim 17 (depends on 11) The method of claim 11, wherein the transforming the RNN further comprises configuring the derivative neural network such that the result of the non-causal operations performed at an instance of the cell is combined with the causal operation performed in respect of that same instance. ### Claim 18 (depends on 1) The method of claim 1, wherein the recurrent neural network comprises a plurality of cells. ### Claim 19 A data processing system for implementing a recurrent neural network (RNN) for operation on a sequence of inputs, the system comprising: a transformation unit configured to receive a representation of the RNN and transform the representation of the RNN into a derivative neural network for operation over a predetermined plurality of inputs of the sequence of inputs, the derivative neural network having one or more state inputs and one or more state outputs and being equivalent to the RNN over a predetermined plurality of steps of the RNN; a hardware accelerator for processing neural networks; and iteration logic configured to iteratively apply the derivative neural network to the sequence of inputs by: causing a sequence of instances of the derivative neural network to be implemented at the hardware accelerator; and providing the one or more state outputs from each representation of the derivative neural network at the hardware accelerator as the one or more state inputs to a subsequent representation of the derivative neural network at the hardware accelerator so as to cause the hardware accelerator to operate the RNN over a sequence of inputs longer than the predetermined plurality of inputs. ### Claim 20 A non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a computer system, cause the computer system to perform a method of implementing in hardware a recurrent neural network (RNN) for operation on a sequence of inputs, each step of the recurrent neural network being for operation on a different input of the sequence, the method comprising: receiving a representation of the RNN; transforming the representation of the RNN into a derivative neural network for operation over a predetermined plurality of inputs of the sequence of inputs, the derivative neural network having one or more state inputs and one or more state outputs and being equivalent to the RNN over a predetermined plurality of steps of the RNN; and iteratively applying the derivative neural network to the sequence of inputs by: implementing a sequence of instances of the derivative neural network in hardware; and providing the one or more state outputs from each instance of the derivative neural network at the hardware as the one or more state inputs to a subsequent instance of the derivative neural network at the hardware so as to operate the RNN over a sequence of inputs longer than the predetermined plurality of inputs.