# US 2018/0322393 A1 · Neural network, corresponding device, apparatus and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0322393 A1
- Document ID: US20180322393A1
- Status: Published application
- Kind: A1
- Application number: 15/965,803
- Filed: 2018-04-27
- Published: 2018-11-08
- Earliest claimed: 2017-05-02
- Assignees: STMicroelectronics S.r.l.
- Applicants: Stmicroelectronics S.R.L.
- Inventors: Danilo Pietro PAU; Marco PIASTRA; Luca CARCANO
- Web page: https://patentgrep.com/patent/US20180322393A1

## CPC

- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/048: Activation functions
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning
- G06N 3/09: Supervised learning
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A neural network includes one layer of neurons including neurons having neuron connections to neurons in the layer and input connections to a network input. The neuron connections and the input connections have respective neuron connection weights and input connection weights. The neurons have neuron responses set by an activation function with activation values and include activation function computing circuits configured for computing current activation values of the activation function as a function of previous activation values of the activation function and current network input values.

## Claims

### Claim 1

A neural network, comprising:
at least one layer of a plurality of neurons, the plurality of neurons including neuron connections and input connections, the neuron connections between neurons in the at least one layer of said plurality of neurons, and the input connections between neurons in the at least one layer of said plurality of connections and a network input, wherein the neuron connections and the input connections have respective neuron connection weights and input connection weights, wherein neurons in the at least one layer of said plurality of neurons have neuron responses set by an activation function with activation values variable over time, at least one layer of said plurality of said plurality of neurons including activation function computing circuits configured to compute current activation values of the activation function as a function of previous activation values of the activation function and current network input values.

### Claim 2 (depends on 1)

The neural network of claim 1, wherein the neuron connections include neuron self-connections.

### Claim 3 (depends on 1)

The neural network of claim 1, wherein said activation function computing circuits comprise:
distance computing blocks arranged to produce a first output indicative of a distance between said current network input and a respective input connection weight, and arranged to produce a second output indicative of a distance between said previous activation value and a respective neuron connection weight; and
an exponential module arranged to apply an exponential function to a sum of said first and second outputs.

### Claim 4 (depends on 3)

The neural network of claim 3, wherein the distance computing modules are configured to compute said distances as Euclidean distances.

### Claim 5 (depends on 3)

The neural network of claim 3, comprising:
dampening modules arranged to apply dampening factors to said first and second outputs summed to provide said sum of said first and second outputs.

### Claim 6 (depends on 3)

The neural network of claim 3, wherein said activation function computing circuits include a leaky integration stage coupled to an output of said exponential module.

### Claim 7 (depends on 6)

The neural network of claim 6, including:
a multiplier arranged to multiply by a gain factor less than unity, the multiplier coupling the output of said exponential module to an input of the leaky integration stage, wherein the leaky integration stage includes a leaky feedback loop with a leak factor which is complementary to unity of said gain factor less than unity.

### Claim 8

A device, including:
a sensor to provide a sensor signal; and
a neural network, the neural network including:
at least one layer of a plurality of neurons, the plurality of neurons including neuron connections and input connections, the neuron connections between neurons in the at least one layer of said plurality of neurons, and the input connections between neurons in the at least one layer of said plurality of connections and a network input, wherein the neuron connections and the input connections have respective neuron connection weights and input connection weights, wherein neurons in the at least one layer of said plurality of neurons have neuron responses set by an activation function with activation values variable over time, at least one layer of said plurality of said plurality of neurons including activation function computing circuits configured to compute current activation values of the activation function as a function of previous activation values of the activation function and current network input values;
an input stage coupled to said sensor and configured to receive said sensor signal as said network input; and
a readout stage to provide a network-processed output signal.

### Claim 9 (depends on 8)

The device of claim 8, wherein the sensor comprises:
an accelerometer coupled to the input stage, the accelerometer configured to provide activity signals, wherein said network-processed output is arranged to include classifications of said activity signals.

### Claim 10 (depends on 9)

The device of claim 9, comprising:
a gyroscope coupled to the accelerometer and arranged to provide activity signals.

### Claim 11 (depends on 8)

The device of claim 8, wherein the device is a wearable computing device.

### Claim 12 (depends on 8)

The device of claim 8, comprising:
a presentation unit arranged to present said network-processed output signal.

### Claim 13 (depends on 12)

The device of claim 12, wherein the presentation unit is further arranged to present an activity classification that classifies the network-processed output signal.

### Claim 14

A method of adaptively setting neuron connection weights and input weights in a self-organizing neural network, comprising:
providing a neural network having at least one layer of a plurality of neurons, the plurality of neurons including neuron connections and input connections, the neuron connections between neurons in the at least one layer of said plurality of neurons, and the input connections between neurons in the at least one layer of said plurality of connections and a network input, wherein the neuron connections and the input connections have, respectively, the neuron connection weights and the input connection weights;
receiving input values, the input values including at least one input value for said input weights and at least one input value for said connection weights;
calculating a distance between said input values and, respectively, said input weights and said connection weights;
applying dampening to the distance calculated, said dampening including:
i) first dampening with a distance decay which is a function of distance to neighboring neurons in said at least one layer and
ii) second learning rate dampening with a time decay which is a function of time; and
calculating updates for said respective neuron connection weights and input weights as a function of said distance calculated with said dampening applied.

### Claim 15 (depends on 14)

The device of claim 14, wherein the function of distance and the function of time are exponential functions.

### Claim 16 (depends on 14)

The method of claim 14, wherein the neural network includes a classification readout stage configured to provide classification of signals that are input to the neural network, the method comprising:
subsequent to adaptively setting said neuron connection weights and input weights:
receiving a set of known input signals at said classification readout stage;
operating said classification readout stage to provide candidate classifications for said known input signals;
comparing said candidate classifications with known classifications for said known input signals; and
correcting the neuron connection weights and input weights in nodes in said classification readout stage of the neural network targeting correspondence of said candidate classifications with said known classifications.

### Claim 17 (depends on 16)

The device of claim 16, wherein said signals that are input to the neural network are signals from at least one of an accelerometer and a gyroscope.

### Claim 18

A non-transitory computer program product, loadable in the memory of at least one computer and including software code portions executable by a processor to perform a method, the method comprising:
providing a self-organizing neural network having at least one layer of a plurality of neurons, the plurality of neurons including neuron connections and input connections, the neuron connections between neurons in the at least one layer of said plurality of neurons, and the input connections between neurons in the at least one layer of said plurality of connections and a network input, wherein the neuron connections and the input connections have, respectively, the neuron connection weights and the input connection weights;
passing sensor signals to the network input of said self-organizing neural network; and
passing a network-processed output signal to a readout stage.

### Claim 19 (depends on 18)

The non-transitory computer program product of claim 18, the method comprising:
setting neuron connection weights and input weights in the self-organizing neural network;
receiving input values, the input values including at least one input value for said input weights and at least one input value for said connection weights;
calculating a distance between said input values and, respectively, said input weights and said connection weights;
applying dampening to the distance calculated, said dampening including:
i) first dampening with a distance decay which is a function of distance to neighboring neurons in said at least one layer and
ii) second learning rate dampening with a time decay which is a function of time; and
calculating updates for said respective neuron connection weights and input weights as a function of said distance calculated with said dampening applied.

### Claim 20 (depends on 19)

The non-transitory computer program product of claim 19, the method comprising:
after setting said neuron connection weights and said input weights:
receiving a set of known input signals at a classification readout stage;
operating said classification readout stage to provide candidate classifications for said set of known input signals;
comparing said candidate classifications with known classifications for said known input signals; and
correcting the neuron connection weights and input weights in nodes in said classification readout stage of the neural network targeting correspondence of said candidate classifications with said known classifications.