# US 2022/0044096 A1 · Number Format Selection in Recurrent Neural Networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044096 A1
- Document ID: US20220044096A1
- Status: Published application
- Kind: A1
- Application number: 17/368,302
- Filed: 2021-07-06
- Published: 2022-02-10
- Earliest claimed: 2020-07-03
- Assignees: Imagination Technologies Limited
- Applicants: Imagination Technologies Limited
- Inventors: Muhammad Asad; Elia Condorelli; James Imber; Cagatay Dikici
- Web page: https://patentgrep.com/patent/US20220044096A1

## CPC

- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/063: using electronic means
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented method of selecting a number format for representing two or more values of a recurrent neural network (RNN) for use in configuring a hardware implementation of the RNN, includes receiving a representation of the RNN; implementing the representation of the RNN as a test neural network for operation on a sequence of test inputs, each step of the test neural network comprising an instance of the two or more values of the RNN; operating the test neural network for a plurality of steps on the sequence of test inputs and collecting statistics for provision to a number format selection algorithm; and applying a number format selection algorithm to the statistics so as to derive a common number format for the plurality of instances of the two or more values of the RNN.

## Claims

### Claim 1

A computer-implemented method of selecting a number format for representing two or more values of a recurrent neural network (RNN) for use in configuring a hardware implementation of the RNN, the method comprising:
receiving a representation of the RNN;
implementing the representation of the RNN as a test neural network for operation on a sequence of test inputs, each step of the test neural network comprising an instance of the two or more values of the RNN;
operating the test neural network for a plurality of steps on the sequence of test inputs and collecting statistics for provision to a number format selection algorithm; and
applying a number format selection algorithm to the statistics so as to derive a common number format for the plurality of instances of the two or more values of the RNN.

### Claim 2 (depends on 1)

The method of claim 1, wherein each step of the test neural network is for operation on a different test input of the sequence and wherein the applying the format selection algorithm comprises applying the number format selection algorithm to the statistics captured over all of the plurality of steps, the common number format being output by the number format selection algorithm.

### Claim 3 (depends on 1)

The method of claim 1, wherein the common number format is a block-configurable number format defined by one or more configurable parameters and wherein the number format selection algorithm is configured to identify a block-configurable number format of a predefined type of block-configurable number format.

### Claim 4 (depends on 1)

The method of claim 1, wherein the applying the number format selection algorithm comprises:
independently identifying, for each of the plurality of steps, a number format for each instance of the two or more values; and
combining the number formats for the plurality of instances of the two or more values so as to derive the common number format for the plurality of instances of the two or more values of the RNN.

### Claim 5 (depends on 4)

The method of claim 4, wherein the number format selection algorithm is configured to identify a block-configurable number format defined by one or more configurable parameters for each instance of the two or more values, and wherein the combining comprises independently combining each of the one or more configurable parameters of the block-configurable number formats identified for each instance of the two or more values so as to define the one or more configurable parameters for the common number format.

### Claim 6 (depends on 1)

The method of claim 1, wherein the plurality of steps is a first predetermined plurality of steps, and wherein the implementing the representation of the RNN as a test neural network comprises transforming the representation of the RNN into a test neural network for operation over the first predetermined plurality of steps, the test neural network being equivalent to the RNN over the first predetermined plurality of steps.

### Claim 7 (depends on 6)

The method of claim 6, wherein the transforming comprises unrolling the RNN over the first predetermined plurality of steps so as to form the test neural network.

### Claim 8 (depends on 6)

The method of claim 6, wherein the test neural network is configured to operate on a predefined plurality of test inputs, the predefined plurality of test inputs being equal in number to the first predetermined plurality of steps.

### Claim 9 (depends on 1)

The method of claim 1, further comprising using the common number formats as the number formats for the respective two or more values in a hardware implementation of the RNN.

### Claim 10 (depends on 9)

The method of claim 9, wherein the hardware implementation of the RNN comprises an implementation of the RNN in hardware formed by:
transforming the representation of the RNN into a derivative neural network for operation over a predetermined plurality of inputs of the sequence of inputs, the derivative neural network having one or more state inputs and one or more state outputs and being equivalent to the RNN over a second predetermined plurality of steps of the RNN; and
iteratively applying the derivative neural network to the sequence of inputs by:
implementing a sequence of instances of the derivative neural network in hardware; and
providing the one or more state outputs from each instance of the derivative neural network at the hardware as the one or more state inputs to a subsequent instance of the derivative neural network at the hardware so as to operate the RNN over a sequence of inputs longer than the predetermined plurality of inputs.

### Claim 11 (depends on 10)

The method of claim 10, wherein the common number format formed for each of the two or more values of the RNN is used as the number format for all of the instances of the two or more values in the derivative neural network.

### Claim 12 (depends on 10)

The method of claim 10, wherein the first predetermined plurality of steps comprises fewer steps than the second predetermined plurality of steps.

### Claim 13 (depends on 10)

The method of claim 10, wherein the RNN comprises one or more cells, each cell arranged to receive a cell state input generated at a preceding step, and the transforming the RNN into the test neural network further comprises, at each cell:
identifying non-causal operations which are for performance without dependence on the state input generated at a preceding step; and
in the derivative neural network, grouping together at least some of the non-causal operations at a plurality of instances of the cell over at least some of the predetermined plurality of steps for processing in parallel at the hardware.

### Claim 14 (depends on 13)

The method of claim 13, wherein the cell comprises causal operations which are for performance in dependence on the cell state input and the transforming the RNN further comprises configuring the test neural network such that the result of the non-causal operations performed at the cell in respect of an input from the sequence of test inputs is combined with the causal operations performed at the cell in respect of that same test input.

### Claim 15 (depends on 13)

The method of claim 13, wherein the two or more values are used in the non-causal operations and the RNN comprises two or more other values which are used in the causal operations, and the applying the number format selection algorithm to the statistics is performed so as to independently derive the common number format for the two or more values of the RNN and a second common number format for the two or more other values of the RNN.

### Claim 16 (depends on 1)

The method of claim 1, wherein the sequence of test inputs comprise exemplary input values selected to represent a typical or expected range of input values to the RNN.

### Claim 17

A data processing system for selecting one or more number formats for representing two or more values of a recurrent neural network (RNN) for use in configuring a hardware implementation of the RNN, the data processing system comprising:
a processor;
control logic configured to implement a representation of the RNN at the processor as a test neural network for operation on a sequence of test inputs, each step of the test neural network comprising an instance of the two or more values of the RNN; and
a format selection unit configured to cause the processor to operate the test neural network for a plurality of steps on the sequence of test inputs and collect statistics for provision to a number format selection algorithm;
wherein the format selection unit is configured to apply the number format selection algorithm to the statistics so as to derive a common number format for the plurality of instances of the two or more values of the RNN.

### Claim 18 (depends on 17)

The data processing system of claim 17, further comprising a hardware accelerator for processing neural networks, wherein the control logic is further configured to cause the representation of the RNN to be implemented at the hardware accelerator using the common number format for the two or more values of the RNN.

### Claim 19 (depends on 18)

The data processing system of claim 18, further comprising:
a transformation unit configured to transform the representation of the RNN derivative neural network for operation over a predetermined plurality of inputs of a sequence of inputs, the derivative neural network having one or more state inputs and one or more state outputs and being equivalent to the RNN over a predetermined plurality of steps of the RNN;
iteration logic configured to, subsequent to the operation of the test neural network at the processor, iteratively apply the derivative neural network to the sequence of inputs by:
causing a sequence of instances of the derivative neural network to be implemented at the hardware accelerator; and
providing the one or more state outputs from each representation of the derivative neural network at the hardware accelerator as the one or more state inputs to a subsequent representation of the derivative neural network at the hardware accelerator so as to cause the hardware accelerator to operate the RNN over a sequence of inputs longer than the predetermined plurality of inputs.

### Claim 20

A non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a computer system, cause the computer system to perform a computer-implemented method of selecting a number format for representing two or more values of a recurrent neural network (RNN) for use in configuring a hardware implementation of the RNN, the method comprising:
receiving a representation of the RNN;
implementing the representation of the RNN as a test neural network for operation on a sequence of test inputs, each step of the test neural network comprising an instance of the two or more values of the RNN;
operating the test neural network for a plurality of steps on the sequence of test inputs and collecting statistics for provision to a number format selection algorithm; and
applying a number format selection algorithm to the statistics so as to derive a common number format for the plurality of instances of the two or more values of the RNN.
