# US 11,308,406 B2 · Method of operating neural networks, corresponding network, apparatus and computer program product

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,406 B2
- Document ID: US11308406B2
- Status: Granted
- Kind: B2
- Application number: 15/877,138
- Filed: 2018-01-22
- Published: 2018-08-02
- Granted: 2022-04-19
- Earliest claimed: 2017-01-27
- Assignees: STMicroelectronics S.r.l.
- Applicants: STMICROELECTRONICS S.r.l.
- Inventors: Valentina Arrigoni; Giuseppe Desoli; Beatrice Rossi; Pasqualina Fragneto
- Web page: https://patentgrep.com/patent/US11308406B2

## CPC

- G06N 3/045: Combinations of networks
- G06N 5/046: Forward inferencing; Production systems
- G06F 7/729: {using representation by a residue number system}
- G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)}
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 7/023: {Learning or tuning the parameters of a fuzzy system}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method of operating neural networks such as convolutional neural networks including, e.g., an input layer, an output layer and at least one intermediate layer between the input layer and the output layer, with the network layers including operating circuits performing arithmetic operations on input data to provide output data. The method includes: selecting a set of operating circuits in the network layers, performing arithmetic operations in operating circuits in the selected set of operating circuits by performing Residue Number System or RNS operations on RNS-converted input data by obtaining RNS output data in the Residue Number System, backward converting from the Residue Number System the RNS output data resulting from the RNS operations.

## Claims

### Claim 1

A method, comprising:
scaling, using a set of operating circuits of a neural network, input data to the set of operating circuits using input scaling factors;
converting, using the set of operating circuits of the neural network, the scaled input data to the set of operating circuits of the neural network to a Residual Number System (RNS) domain from one or more other domains, the neural network having a plurality of layers;
performing arithmetic operations in the set of operating circuits in the RNS domain on the converted data;
converting, using the set of operating circuits of the neural network, output results of the arithmetic operations of the set of operating circuits from the RNS domain to the one or more other domains;
scaling the converted output results in the one or more other domains using output scaling factors complementary to the input scaling factors, generating scaled output results, wherein the input scaling factors are selected to determining a range of represented values in the RNS domain; and
performing a classification operation based on the scaled output results.

### Claim 2 (depends on 1)

The method of claim 1, comprising:
receiving at least some of the input data from one or more sensors; and
generating one or more control signals based on results of the classification operation.

### Claim 3 (depends on 2)

The method of claim 2 wherein the one or more control signals include:
control parameters;
control signals;
model parameters;
warning signals; or
combinations thereof.

### Claim 4 (depends on 1)

The method of claim 1, comprising:
selecting the set of operating circuits.

### Claim 5 (depends on 1)

The method of claim 1 wherein the set of operating circuits perform arithmetic operations selected out of multiplications, additions and subtractions.

### Claim 6 (depends on 1)

The method of claim 1 wherein the set of operating circuits includes operating circuits performing operations homogeneous with respect to a first set and a second set of input data, the method including converting both said first set and said second set of input data to the RNS domain.

### Claim 7 (depends on 1)

The method of claim 1 wherein the set of operating circuits includes operating circuits performing operations homogeneous with respect to a first set and a second set of input data, the method including:
converting the first set of input data to the RNS domain; and
retrieving the second set of input data from a memory.

### Claim 8 (depends on 1)

The method of claim 1, comprising:
setting input scaling factors and output scaling factors for each layer of the plurality of layers.

### Claim 9 (depends on 1)

The method of claim 1 wherein the one or more other domains includes an integer domain.

### Claim 10 (depends on 1)

The method of claim 1, comprising:
rounding scaled input data prior to converting the scaled input data to the RNS domain.

### Claim 11 (depends on 1)

The method of claim 1, comprising:
using a same set of scaling factors for a plurality of the set of operating circuits;
setting to unity scaling factors for at least a part of the set of operating circuits;
using a power-of-two scaling factor for at least one of the set of operating circuits, wherein scaling includes shifting of binary data; or
combinations thereof.

### Claim 12 (depends on 1)

The method of claim 1, comprising setting the respective scaling factors using a look-up-table.

### Claim 13 (depends on 1)

The method of claim 1, including performing at least one arithmetic operation in a RNS domain with power-of-two moduli.

### Claim 14

A neural network, comprising:
one or more inputs, which, in operation, receive input data;
one or more outputs, which, in operation, output one or more output signals; and
a plurality of network layers coupled between the one or more inputs and the one or more outputs, and which, in operation, perform classification operations, the performing including generating the one or more output signals based on the input data, the plurality of network layers including a set of operating circuits, which, in operation:
scale respective sets of data using input scaling factors;
convert scaled sets of data to a Residual Number System (RNS) domain from one or more other domains, wherein input scaling factors applied to a set of data determine a range of the RNS domain;
perform arithmetic operations in the RNS domain on the converted data;
convert respective results of the arithmetic operations from the RNS domain to the one or more other domains; and
scale the converted results using output scaling factors complementary to the input scaling factors.

### Claim 15 (depends on 14)

The neural network of claim 14 wherein,
the input data comprises sensor data; and
the output signals comprise control signals.

### Claim 16 (depends on 14)

The neural network of claim 14 wherein the neural network, in operation, selects the set of operating circuits.

### Claim 17 (depends on 16)

The neural network of claim 16 wherein the set of operating circuits, in operation, perform arithmetic operations selected out of multiplications, additions and subtractions.

### Claim 18 (depends on 14)

The neural network of claim 14 wherein at least some of the set of operating circuits, in operation:
set input scaling factors and output scaling factors for each layer of the plurality of layers.

### Claim 19 (depends on 14)

The neural network of claim 14 wherein the set of operating circuits, in operation, round at least some of the scaled input data prior to converting the at least some of the scaled input data to the RNS domain.

### Claim 20

A system, comprising:
circuitry, which, in operation, generates source data; and
a neural network coupled to the circuitry, wherein the neural network, in operation, performs classification operations, the performing classification operations including generating one or more output signals based on the source data and including a set of operating circuits, wherein the operating circuits, in operation:
scale respective sets of data using input scaling factors;
convert scaled sets of data to a Residual Number System (RNS) domain from one or more other domains, wherein input scaling factors applied to a set of data determine a range of the RNS domain;
perform arithmetic operations in the RNS domain on the converted data; and
convert respective results of the arithmetic operations from the RNS domain to the one or more other domains; and
scale the converted results using output scaling factors complementary to the input scaling factors.

### Claim 21 (depends on 20)

The system of claim 20 wherein,
the circuitry comprises one or more sensors; and
the output signals comprise control signals.

### Claim 22 (depends on 20)

The system of claim 20 wherein at least some of the set of operating circuits, in operation:
set input scaling factors and output scaling factors for each layer of the plurality of layers.

### Claim 23 (depends on 20)

The system of claim 20 wherein the set of operating circuits, in operation, round at least some of the scaled input data prior to converting the at least some of the scaled input data to the RNS domain.

### Claim 24

A non-transitory computer-readable medium whose contents configure a set of operating circuits of a neural network having a plurality of layers to perform a method, the method comprising:
scaling input data to the set of operating circuits using input scaling factors;
converting scaled input data to the set of operating circuits of the neural network to a Residual Number System (RNS) domain from one or more other domains, the input scaling factors determining a range of represented values in the RNS domain;
performing arithmetic operations in the set of operating circuits in the RNS domain on the converted data;
converting output results of the arithmetic operations of the set of operating circuits from the RNS domain to the one or more other domains;
scaling the converted output results in the one or more other domains using output scaling factors complementary to the input scaling factors, generating scaled output results; and
performing a classification operation based on the scaled output results.

### Claim 25 (depends on 24)

The non-transitory computer-readable medium of claim 24 wherein the method comprises:
receiving at least some of the input data from one or more sensors; and
generating one or more control signals based on the converted output results.

### Claim 26 (depends on 25)

The non-transitory computer-readable medium of claim 25 wherein the method comprises:
selecting the set of operating circuits.

### Claim 27 (depends on 24)

The non-transitory computer-readable medium of claim 24 wherein the method comprises:
setting input scaling factors and output scaling factors for each layer of the plurality of layers.

### Claim 28 (depends on 24)

The non-transitory computer-readable medium of claim 24 wherein the method comprises:
rounding at least some of the scaled input data prior to converting the at least some of the scaled input data to the RNS domain.