# US 2019/0378007 A1 · Characterizing activity in a recurrent artificial neural network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0378007 A1
- Document ID: US20190378007A1
- Status: Published application
- Kind: A1
- Application number: 16/004,635
- Filed: 2018-06-11
- Published: 2019-12-12
- Earliest claimed: 2018-06-11
- Assignees: Inait SA
- Applicants: Inait SA
- Inventors: Henry Markram; Ran Levi; Kathryn Pamela Hess Bellwald
- Web page: https://patentgrep.com/patent/US20190378007A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/086: using evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 3/045: Combinations of networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for characterizing activity in a recurrent artificial neural network. In one aspect, a method can include characterizing activity in an artificial neural network. The method is performed by data processing apparatus and can include identifying clique patterns of activity of the artificial neural network. The clique patterns of activity can enclose cavities.

## Claims

### Claim 1

A method comprising:
characterizing activity in an artificial neural network, the method performed by data processing apparatus and comprising identifying clique patterns of activity of the artificial neural network, wherein the clique patterns of activity enclose cavities.

### Claim 2 (depends on 1)

The method of claim 1, wherein the method further comprises defining a plurality of windows of time during which the activity of the artificial neural network is responsive to an input into the artificial neural network, wherein the clique patterns of activity are identified in each of the pluralities of windows of time.

### Claim 3 (depends on 2)

The method of claim 2, wherein the method further comprises identifying a first window of time within the plurality of windows of time based on a distinguishable likelihood of the clique patterns of activity occurring during the first window.

### Claim 4 (depends on 1)

The method of claim 1, wherein identifying clique patterns comprises identifying directed cliques of activity.

### Claim 5 (depends on 4)

The method of claim 4, wherein identifying directed cliques comprises discarding or ignoring lower dimensional directed cliques that are present in higher dimensional directed cliques.

### Claim 6 (depends on 1)

The method of any one of claim 1, further comprising:
classifying the clique patterns into categories; and
characterizing the activity according to the number of occurrences of the clique patterns in respective of the categories.

### Claim 7 (depends on 6)

The method of claim 6, wherein classifying the clique patterns comprises classifying the clique patterns according to a number of points within each clique pattern.

### Claim 8 (depends on 1)

The method of claim 1, further comprising outputting a binary sequence of zeros and ones from the recurrent artificial neural network, wherein each digit in the sequence represents whether or not a respective pattern of activity is present in the artificial neural network.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
structuring the artificial neural network, comprising
reading the digits output from the artificial neural network, and
evolving the structure of the artificial neural network, wherein evolving the structure of the artificial neural network comprises:
iteratively changing the structure,
characterizing the complexity of patterns of activity in the changed structure, and
using the characterization of the complexity of the pattern as an indication of whether the changed structure is desirable.

### Claim 10 (depends on 1)

The method of claim 1, wherein:
the artificial neural network is a recurrent artificial neural network; and
the method further comprises:
identifying decision moments in the recurrent artificial neural network based on the determination of the complexity of patterns of activity in the recurrent artificial neural network, the identification of decision moments comprising
determining a timing of activity having a complexity that is distinguishable from other activity that is responsive to the input, and
identifying the decision moments based on the timing of the activity that has the distinguishable complexity.

### Claim 11 (depends on 10)

The method of claim 10, further comprising inputting a data stream into the recurrent artificial neural network and identifying the clique patterns of activity during the input of the data stream.

### Claim 12 (depends on 1)

The method of claim 1, further comprising estimating whether the activity is responsive to the input into the artificial neural network, the estimating comprising:
estimating that relatively simpler patterns of activity relatively soon after the input event are responsive the input but that relatively more complex patterns of activity relatively soon after the input event are not responsive the input; and
estimating that relatively more complex patterns of activity relatively later after the input event are responsive the input but that relatively simpler patterns of activity relatively later after the input event are not responsive the input.

### Claim 13

A system comprising one or more computers operable to perform operations comprising:
characterizing activity in an artificial neural network, comprising identifying clique patterns of activity of the artificial neural network, wherein the clique patterns of activity enclose cavities.

### Claim 14 (depends on 13)

The system of claim 13, wherein the operations further comprise defining a plurality of windows of time during which the activity of the artificial neural network is responsive to an input into the artificial neural network, wherein the clique patterns of activity are identified in each of the pluralities of windows of time.

### Claim 15 (depends on 14)

The system of claim 14, wherein the operations further comprise identifying a first window of time within the plurality of windows of time based on a distinguishable likelihood of the clique patterns of activity occurring during the first window.

### Claim 16 (depends on 14)

The system of claim 14, wherein identifying clique patterns comprises discarding or ignoring lower dimensional directed cliques that are present in higher dimensional directed cliques.

### Claim 17 (depends on 13)

The system of claim 13, wherein the operations further comprise:
structuring the artificial neural network, comprising
reading the digits output from the artificial neural network, and
evolving the structure of the artificial neural network; wherein evolving the structure of the artificial neural network comprises:
iteratively changing the structure,
characterizing the complexity of patterns of activity in the changed structure, and
using the characterization of the complexity of the pattern as an indication of whether the changed structure is desirable.

### Claim 18 (depends on 13)

The system of claim 13, wherein:
the artificial neural network is a recurrent artificial neural network; and
the operations further comprise:
identifying decision moments in the recurrent artificial neural network based on the determination of the complexity of patterns of activity in the recurrent artificial neural network, the identification of decision moments comprising
determining a timing of activity having a complexity that is distinguishably from other activity that is responsive to the input, and
identifying the decision moments based on the timing of the activity that has the distinguishable complexity.

### Claim 19 (depends on 18)

The system of claim 18, wherein the operations further comprise inputting a data stream into the recurrent artificial neural network and identifying the clique patterns of activity during the input of the data stream.

### Claim 20 (depends on 13)

The system of claim 13, wherein the operations further comprise estimating whether the activity is responsive to the input into the artificial neural network, the estimating comprising:
estimating that relatively simpler patterns of activity relatively soon after the input event are responsive the input but that relatively more complex patterns of activity relatively soon after the input event are not responsive the input; and
estimating that relatively more complex patterns of activity relatively later after the input event are responsive the input but that relatively simpler patterns of activity relatively later after the input event are not responsive the input.
