# US 12,217,161 B2 · Convolutional neural network hardware configuration

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,161 B2
- Document ID: US12217161B2
- Status: Granted
- Kind: B2
- Application number: 17/498,618
- Filed: 2021-10-11
- Published: 2022-01-27
- Granted: 2025-02-04
- Earliest claimed: 2016-05-03
- Assignees: Imagination Technologies Limited
- Applicants: Imagination Technologies Limited
- Inventors: Clifford Gibson; James Imber
- Web page: https://patentgrep.com/patent/US12217161B2

## CPC

- G06N 3/063: using electronic means
- G06F 7/00: Methods or arrangements for processing data by operating upon the order or content of the data handled (logic circuits H03K19/00)
- G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/048: Activation functions
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method of configuring a hardware implementation of a Convolutional Neural Network (CNN), the method comprising: determining, for each of a plurality of layers of the CNN, a first number format for representing weight values in the layer based upon a distribution of weight values for the layer, the first number format comprising a first integer of a first predetermined bit-length and a first exponent value that is fixed for the layer; determining, for each of a plurality of layers of the CNN, a second number format for representing data values in the layer based upon a distribution of expected data values for the layer, the second number format comprising a second integer of a second predetermined bit-length and a second exponent value that is fixed for the layer; and storing the determined number formats for use in configuring the hardware implementation of a CNN.

## Claims

### Claim 1

A computer-implemented method of configuring a hardware implementation of a Convolutional Neural Network (CNN), the method comprising:
receiving, in a computer system, parameters of a CNN having a plurality of layers, said parameters including weight values and expected data values;
determining, in the computer system, for each of a plurality of layers of the CNN, a first number format for representing weight values in the layer based upon a distribution of weight values for the layer, the first number format comprising a first integer of a first predetermined bit-length and a first exponent value that is fixed for the layer;
determining, in the computer system, for each of a plurality of layers of the CNN, a second number format for representing data values in the layer based upon a distribution of expected data values for the layer, the second number format comprising a second integer of a second predetermined bit-length and a second exponent value that is fixed for the layer;
storing in a computer readable storage medium the determined number formats for use in configuring the hardware implementation of a CNN; and
tuning the weight values, the tuning comprising:
selecting, for each of a plurality of layers, a subset of the weight values in that layer, wherein the subset is less than all of the weight values in that layer,
quantising, for each of a plurality of layers, the selected subset of weight values in that layer in accordance with the first number format for that layer to generate modified weight values in that layer, wherein the modified weight values in that layer are less than all of the weight values in that layer, and
training the CNN using the modified weight values to generate tuned weight values.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first and second predetermined bit-lengths are each fixed across a plurality of layers of the CNN.

### Claim 3 (depends on 1)

The method of claim 1, further comprising converting the weight values for each of the plurality of layers of the CNN into the first number format and converting input data to the CNN into the second number format, wherein the converting comprises performing quantisation based upon the first and second number formats.

### Claim 4 (depends on 3)

The method of claim 3, wherein the quantisation is performed based upon nearest-neighbour rounding.

### Claim 5 (depends on 3)

The method of claim 3, further comprising storing the converted weight values for each of the plurality of layers of the CNN and data values to the CNN in a memory accessible by the hardware implementation of the CNN or in one or more buffers in the hardware implementation of the CNN.

### Claim 6 (depends on 1)

The method of claim 1, further comprising storing the weight values for each of the plurality of layers of the CNN, the storing comprising:
storing, for each weight value, a value representing the first integer of that weight value; and
storing, for each layer, a value representing the first exponent value for that layer.

### Claim 7 (depends on 1)

The method of claim 1, further comprising storing the data values for each of the plurality of layers of the CNN, the storing comprising:
storing, for each data value, a value representing the second integer of that data value; and
storing, for each layer, a value representing the second exponent value for that layer.

### Claim 8 (depends on 1)

The method of claim 1, wherein the weight values in each layer of the CNN are generated by training the CNN using a training data set and wherein the generated weight values are in a floating-point format.

### Claim 9 (depends on 8)

The method of claim 8, wherein the distribution of expected data values is derived from the training data set.

### Claim 10 (depends on 1)

The method of claim 1, further comprising determining the first exponent value for each layer by:
identifying a bit index that represents a range of values based upon the distribution of weight values for that layer;
right-shifting the position of the bit index from the identified bit index a number of bits based upon the first predetermined bit-length to identify a first shifted bit index; and
allocating the value of the first shifted bit index as the first exponent value.

### Claim 11 (depends on 1)

The method of claim 1, further comprising determining the second exponent value for each layer by:
identifying a bit index that represents a range of values based upon the distribution of expected data values for that layer;
right-shifting the position of the bit index from the identified bit index a number of bits based upon the second predetermined bit-length to identify a second shifted bit index; and
allocating the value of the second shifted bit index as the second exponent value.

### Claim 12

A computer system for configuring a hardware implementation of a Convolutional Neural Network (CNN), the computer system comprising:
hardware logic configured to determine, for each of a plurality of layers of the CNN, a first number format for representing weight values in the layer based upon a distribution of weight values for the layer, the first number format comprising a first integer of a first predetermined bit-length and a first exponent value that is fixed for the layer;
hardware logic configured to determine, for each of a plurality of layers of the CNN, a second number format for representing data values in the layer based upon a distribution of expected data values for the layer, the second number format comprising a second integer of a second predetermined bit-length and a second exponent value that is fixed for the layer;
the computer system further configured to store the determined number formats in a computer readable storage medium for use in configuring the hardware implementation of a CNN; and
tune the weight values, wherein tuning the weight values comprises:
selecting, for each of a plurality of layers, a subset of the weight values in that layer, wherein the subset is less than all of the weight values in that layer,
quantising, for each of a plurality of layers, the selected subset of weight values in that layer in accordance with the first number format for that layer to generate modified weight values in that layer, wherein the modified weight values in that layer are less than all of the weight values in that layer, and
training the CNN using the modified weight values to generate tuned weight values.

### Claim 13 (depends on 12)

The computer system of claim 12, wherein the first and second predetermined bit-lengths are each fixed across a plurality of layers of the CNN.

### Claim 14 (depends on 12)

The computer system of claim 12, wherein the computer system is configured to convert the weight values for each of the plurality of layers of the CNN into the first number format and convert input data to the CNN into the second number format, wherein the conversion comprises performing quantisation based upon the first and second number formats.

### Claim 15 (depends on 14)

The computer system of claim 14, wherein the quantisation is performed based upon nearest-neighbour rounding.

### Claim 16 (depends on 14)

The computer system of claim 14, wherein the computer system is configured to store the converted weight values for each of the plurality of layers of the CNN and data values to the CNN in a memory accessible by the hardware implementation of the CNN or in one or more buffers in the hardware implementation of the CNN.

### Claim 17 (depends on 12)

The computer system of claim 12, wherein the computer system is configured to store the weight values for each of the plurality of layers of the CNN, wherein storing the weight values comprises:
storing, for each weight value, a value representing the first integer of that weight value; and
storing, for each layer, a value representing the first exponent value for that layer.

### Claim 18 (depends on 12)

The computer system of claim 12, wherein the computer system is configured to store the data values for each of the plurality of layers of the CNN, wherein storing the data values comprises:
storing, for each data value, a value representing the second integer of that data value; and
storing, for each layer, a value representing the second exponent value for that layer.

### Claim 19 (depends on 12)

The computer system of claim 12, wherein the weight values in each layer of the CNN are generated by training the CNN using a training data set and wherein the generated weight values are in a floating-point format.

### Claim 20

A non-transitory computer readable storage medium having stored thereon computer executable instructions that when executed cause at least one processor to configure a hardware implementation of a Convolutional Neural Network (CNN), by:
determining, for each of a plurality of layers of the CNN, a first number format for representing weight values in the layer based upon a distribution of weight values for the layer, the first number format comprising a first integer of a first predetermined bit-length and a first exponent value that is fixed for the layer;
determining, for each of a plurality of layers of the CNN, a second number format for representing data values in the layer based upon a distribution of expected data values for the layer, the second number format comprising a second integer of a second predetermined bit-length and a second exponent value that is fixed for the layer;
storing the determined number formats in a computer readable storage medium for use in configuring the hardware implementation of a CNN; and
tuning the weight values, the tuning comprising:
selecting, for each of a plurality of layers, a subset of the weight values in that layer, wherein the subset is less than all of the weight values in that layer,
quantising, for each of a plurality of layers, the selected weight values in that layer in accordance with the first number format for that layer to generate modified weight values in that layer, wherein the selected weight values in that layer are less than all of the weight values in that layer, and
training the CNN using the modified weight values to generate tuned weight values.