# US 2024/0202706 A1 · Methods and systems for selecting quantisation parameters for deep neural networks using back-propagation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202706 A1
- Document ID: US20240202706A1
- Status: Published application
- Kind: A1
- Application number: 18/594,450
- Filed: 2024-03-04
- Published: 2024-06-20
- Earliest claimed: 2018-12-21
- Assignees: Imagination Technologies Limited
- Applicants: Imagination Technologies Limited
- Inventors: Szabolcs Csefalvay
- Web page: https://patentgrep.com/patent/US20240202706A1

## CPC

- G06N 3/063: using electronic means
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/048: Activation functions
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/06: Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06Q 20/3672: {initialising or reloading thereof}
- G06Q 20/389: {Keeping log of transactions for guaranteeing non-repudiation of a transaction}
- G06Q 30/0278: {Product appraisal}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Methods and systems for identifying quantisation parameters for a Deep Neural Network (DNN). The method includes determining an output of a model of the DNN in response to training data, the model of the DNN comprising one or more quantisation blocks configured to transform a set of values input to a layer of the DNN prior to processing the set of values in accordance with the layer, the transformation of the set of values simulating quantisation of the set of values to a fixed point number format defined by one or more quantisation parameters; determining a cost metric of the DNN based on the determined output and a size of the DNN based on the quantisation parameters; back-propagating a derivative of the cost metric to one or more of the quantisation parameters to generate a gradient of the cost metric for each of the one or more quantisation parameters; and adjusting one or more of the quantisation parameters based on the gradients.

## Claims

### Claim 1

A computer-implemented method to configure hardware to implement a Deep Neural Network (DNN), the method comprising, in at least one processor:
(a) determining an output of a model of the DNN in response to training data, the model of the DNN comprising a quantisation block configured to transform a set of values input to a layer of the DNN prior to the model processing the set of values in accordance with the layer, the transformation of the set of values simulating quantisation of the set of values to a fixed point number format defined by one or more quantisation parameters, the one or more quantisation parameters comprising a bit width;
(b) determining a cost metric of the DNN based on the determined output and a size of the DNN based on the bit width;
(c) back-propagating a derivative of the cost metric to the bit width to generate a gradient of the cost metric for the bit width;
(d) adjusting the bit width based on the gradient of the cost metric for the bit width to generate an adjusted bit width; and
(e) configuring the hardware to implement the DNN using the adjusted bit width by configuring the hardware to receive and process the set of values in accordance with the adjusted bit width.

### Claim 2 (depends on 1)

The method of claim 1, wherein adjusting the bit width based on the gradient of the cost metric for the bit width comprises adding a value to the bit width based on the gradient of the cost metric for the bit width to generate a modified bit width.

### Claim 3 (depends on 2)

The method of claim 2, wherein adjusting the bit width based on the gradient of the cost metric for the bit width further comprises quantising the modified bit width to a nearest integer in a set of integers, or simulating the quantisation of the modified bit width to the nearest integer in the set of integers.

### Claim 4 (depends on 3)

The method of claim 3, wherein quantising the modified bit width to the nearest integer in the set of integers comprises rounding the modified bit width to the nearest integer in the set of integers.

### Claim 5 (depends on 3)

The method of claim 3, wherein simulating the quantisation of the modified bit width to the nearest integer in the set of integers comprises adding a random number between −a and a to the modified bit width to generate an addition result, wherein a is half a distance between a closest integer of the set of integers that is lower than the modified bit width and a closest integer of the set of integers that is higher than the modified bit width.

### Claim 6 (depends on 5)

The method of claim 5, wherein simulating the quantisation of the modified bit width to the nearest integer of the set of integers further comprises rounding the addition result to the nearest integer in the set of integers.

### Claim 7 (depends on 3)

The method of claim 3, wherein simulating the quantisation of the modified bit width to the nearest integer of the set of integers comprises:
selecting a highest integer in the set of integers that is less than or equal to the modified bit width to generate a floored modified bit width; and
adding a random value between 0 and c to the floored modified bit width, wherein c is a distance between a closest integer in the set of integers that is lower than the modified bit width and a closest integer in the set of integers that is higher than the modified bit width.

### Claim 8 (depends on 1)

The method of claim 1, further comprising repeating (a), (b), (c) and (d) with the adjusted bit width and/or until the cost metric converges.

### Claim 9 (depends on 8)

The method of claim 8, wherein adjusting the bit width based on the gradient of the cost metric for the bit width comprises adding a value to the bit width based on the gradient of the cost metric for the bit width to generate a modified bit width; and adjusting the bit width in a subsequent iteration of (d) comprises adding a value to the modified bit width of a previous iteration based on the gradient of the cost metric for the bit width.

### Claim 10 (depends on 9)

The method of claim 9, wherein the modified bit width is in a floating point number format.

### Claim 11 (depends on 1)

The method of claim 1, wherein the fixed point number format is defined by a plurality of quantisation parameters that comprises the bit width and an exponent.

### Claim 12 (depends on 11)

The method of claim 11, wherein the quantisation block is configured to transform a value of the set of values input to the layer by:
thresholding the value to a range of representable numbers for the fixed point number format for the quantisation block to generate a thresholded value, and rounding the thresholded value to the nearest expth power of 2, wherein exp is the exponent of the fixed point number format for the quantisation block; or
rounding the value to the nearest expth power of 2 to generate a rounded value, and thresholding the rounded value to a range of representable numbers for the fixed point number format for the quantisation block.

### Claim 13 (depends on 12)

The method of claim 12, wherein the quantisation block is further configured to transform the value of the set of values input to the layer by adding a random number between −a and a to one of the value, the thresholded value and the rounded value, wherein a is half a distance between representable numbers in the fixed point number format for the quantisation block.

### Claim 14 (depends on 1)

The method of claim 1, wherein the quantisation block is configured to transform a value of the set of values input to the layer by:
(i) thresholding the value to a range of representable numbers for the fixed point number format for the quantisation block to generate a thresholded value; and adding a random number between −a and a to the thresholded value, wherein a is half a distance between representable numbers in the fixed point number format for the quantisation block; or
(ii) adding a random number between −a and a to the value to generate a randomized value; and thresholding the randomized value to a range of representable numbers for the fixed point number format for the quantisation block.

### Claim 15 (depends on 1)

The method of claim 1, wherein the quantisation block is configured to transform a value of the set of values input to the layer by:
thresholding the value to a range of representable numbers for the fixed point number format for the quantisation block to generate a thresholded value;
determining a floor of the thresholded value to generate a floored value; and
adding a random value between 0 and c to the floored value, wherein c is a distance between representable numbers in the fixed point number format for the quantisation block.

### Claim 16 (depends on 1)

The method of claim 1, wherein the DNN comprises at least one layer in which the input data for the layer is processed in accordance with a plurality of weights, and the method further comprises:
back-propagating the cost metric to one or more weights of the plurality of weights to generate a gradient of the cost metric for each of the one or more weights; and
adjusting each of the one or more weights based on the gradient of the cost metric for that weight.

### Claim 17 (depends on 1)

The method of claim 1, further comprising outputting the adjusted bit width for use in configuring hardware logic to implement the DNN.

### Claim 18 (depends on 1)

The method of claim 1, wherein the hardware is a DNN accelerator.

### Claim 19 (depends on 1)

A non-transitory computer readable storage medium having stored thereon computer readable instructions that, when executed at a computer system, cause the computer system to perform the method as set forth in claim 1.

### Claim 20

A computing-based device to configure hardware to implement a Deep Neural Network (DNN), the computing-based device comprising:
at least one processor; and
memory coupled to the at least one processor, the memory comprising:
computer readable code that when executed by the at least one processor causes the at least one processor to:
(a) determine an output of a model of the DNN in response to training data, the model of the DNN comprising a quantisation block configured to transform a set of values input to a layer of the DNN prior to the model processing the set of values in accordance with the layer, the transformation of the set of values simulating quantisation of the set of values to a fixed point number format defined by one or more quantisation parameters, the one or more quantisation parameters comprising a bit width,
(b) determine a cost metric of the DNN based on the output and a size of the DNN based on the bit width,
(c) back-propagate a derivative of the cost metric to the bit width to generate a gradient of the cost metric for the bit width,
(d) adjust the bit width based on the gradient of the cost metric for the bit width to generate an adjusted bit width, and
(e) configure the hardware to implement the DNN using the adjusted bit width by configuring the hardware to receive and process the set of values in accordance with the adjusted bit width.
