# US 2019/0318224 A1 · Efficient and scalable systems for calculating neural network connectivity in an event-driven way

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318224 A1
- Document ID: US20190318224A1
- Status: Published application
- Kind: A1
- Application number: 16/453,876
- Filed: 2019-06-26
- Published: 2019-10-17
- Earliest claimed: 2014-10-01
- Assignees: Thalchemy Corporation
- Applicants: Thalchemy Corporation
- Inventors: Mikko H. Lipasti; Andrew Nere; Atif Hashmi; John F. Wakerly
- Web page: https://patentgrep.com/patent/US20190318224A1

## CPC

- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/08: Learning methods
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods achieving scalable and efficient connectivity in neural algorithms by re-calculating network connectivity in an event-driven way are disclosed. The disclosed solution eliminates the storing of a massive amount of data relating to connectivity used in traditional methods. In one embodiment, a deterministic LFSR is used to quickly, efficiently, and cheaply re-calculate these connections on the fly. An alternative embodiment caches some or all of the LFSR seed values in memory to avoid sequencing the LFSR through all states needed to compute targets for a particular active neuron. Additionally, connections may be calculated in a way that generates neural networks with connections that are uniformly or normally (Gaussian) distributed.

## Claims

### Claim 1

A scalable system for recalculating, in an event-driven manner, property parameters including connectivity parameters of a neural network, the system comprises:
an input component that receives a time varying input signal;
a storage component for storing the property parameters of the neural network;
a state machine capable of recalculating property parameters of the neural network, wherein the property parameters include connectivity among neurons of the neural network; and
an output component that generates output signals reflective of the calculated property parameters of the neural network and the input signal.

### Claim 2 (depends on 1)

The system of claim 1, wherein the state machine is capable of generating a unique identifying number for each neuron in the neural network.

### Claim 3 (depends on 1)

The system of claim 1, wherein the state machine comprises a Linear Feedback Shift Register (LFSR).

### Claim 4 (depends on 3)

The system of claim 3, wherein the LFSR is configured to generate certain property parameters including connectivity.

### Claim 5 (depends on 1)

The system of claim 1, wherein the input component converts the received time varying input signal into a sequence of spikes.

### Claim 6 (depends on 1)

The system of claim 1, wherein the state machine recalculates connectivity of a neuron currently being evaluated using a predefined initial value corresponding to the neuron currently being evaluated only when the neuron currently being evaluated fires in response to the input signal.

### Claim 7 (depends on 1)

The system of claim 1, wherein the storage component includes a cache for storing predefined initial values used by the state machine for recalculating certain property parameters.

### Claim 8 (depends on 1)

The system of claim 1, wherein the state machine is also used to generate a connection type for each neuron of the network.

### Claim 9 (depends on 1)

The system of claim 1, wherein the property parameters further include neural delays of each neuron in the network.

### Claim 10 (depends on 1)

The system of claim 1, further comprises:
a plurality of processing elements, wherein each processing element has a state machine and is capable of calculating property parameters of a subset of neurons of the neural network.

### Claim 11

A computer-implemented method for recalculating network property parameters of a neural network including connectivity parameters in an event-driven manner, the method comprises:
initializing property parameters of the neural network;
receiving, at an evaluating neuron of the neural network, a neural input corresponding to a time varying input signal to the neural network;
recalculating by a state machine of the neural network at least some of the property parameters of the evaluating neuron, wherein the property parameters are random but determined after initialization;
determining whether the evaluating neuron is to generate a neural output to its target neurons in the neural network; and
if the evaluating neuron is determined to generate a neural output to its target neurons in the neural network, propagating the output of the evaluating neuron to its target neurons.

### Claim 12 (depends on 11)

The method of claim 11, wherein the property parameters of the neural network include one or more parameters of the group consisting of maximum number of neurons in the neural network, one or more random number generation seed values, neural axonal and dendritic delay values, positive connectivity strength values, negative connectivity strength values, neuron refractory period, decay rate of neural membrane potential, neuron membrane potential, and neuron membrane leakage parameter.

### Claim 13 (depends on 11)

The method of claim 11, wherein the determining comprises:
calculating current membrane potential of the evaluating neuron based on the neural input and the connectivity parameters of the evaluating neuron;
comparing the calculated membrane potential to a firing threshold value of the evaluating neuron; and
reporting that the evaluating neuron is to generate an output if the calculated membrane potential exceeds the firing threshold value.

### Claim 14 (depends on 11)

The method of claim 11, wherein the recalculating comprises:
using a pseudo-random number generator with a pre-defined start value to calculate the property parameters.

### Claim 15 (depends on 14)

The method of claim 14, wherein the pseudo-random number generator comprises a Linear Feedback Shift Register (LFSR).

### Claim 16 (depends on 11)

The method of claim 11, wherein the recalculating comprises:
retrieving a stored pre-defined initial value corresponding to the evaluating neuron; and
calculating connectivity parameters of the evaluating neuron using the retrieved seed value.

### Claim 17 (depends on 11)

The method of claim 11, wherein the recalculating comprises calculating connectivity of a neuron currently being evaluated only when the neuron currently being evaluated fires in response to the input signal.

### Claim 18 (depends on 11)

The method of claim 11, further comprises retrieving an initial value of the state machine for recalculating the connectivity parameters of the evaluating neuron from a cache coupled to the state machine.

### Claim 19 (depends on 11)

The method of claim 11, further comprises:
maintaining a list of future firing neurons, wherein the updating at each time step is conducted only on neurons identified on the list;
for each target neuron of a neuron that fires at a current time step, comparing the current membrane potential of that target neuron to a corresponding predefined firing threshold of that target neuron;
adding an identity of a target neuron to the list of future firing neurons if the current membrane potential of that target neuron exceeds the corresponding predefined firing threshold; and
removing an identity of a target neuron from the list of future firing neurons if the current membrane potential of that target neuron is below the corresponding predefined firing threshold.

### Claim 20 (depends on 11)

The method of claim 11, further comprises:
taking sum of the state machine results to form a uniform distribution; and
adding an offset to center the normalized distribution in calculating addresses of the target neurons.