# US 2013/0159229 A1 · Multi-modal neural network for universal, online learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0159229 A1
- Document ID: US20130159229A1
- Status: Published application
- Kind: A1
- Application number: 13/325,321
- Filed: 2011-12-14
- Published: 2013-06-20
- Earliest claimed: 2011-12-14
- Assignees: International Business Machines Corporation
- Inventors: Dharmendra S. Modha
- Web page: https://patentgrep.com/patent/US20130159229A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0499: Feedforward networks
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 3/02: Neural networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

In one embodiment, the present invention provides a neural network comprising multiple modalities. Each modality comprises multiple neurons. The neural network further comprises an interconnection lattice for cross-associating signaling between the neurons in different modalities. The interconnection lattice includes a plurality of perception neuron populations along a number of bottom-up signaling pathways, and a plurality of action neuron populations along a number of top-down signaling pathways. Each perception neuron along a bottom-up signaling pathway has a corresponding action neuron along a reciprocal top-down signaling pathway. An input neuron population configured to receive sensory input drives perception neurons along a number of bottom-up signaling pathways. A first set of perception neurons along bottom-up signaling pathways drive a first set of action neurons along top-down signaling pathways. Action neurons along a number of top-down signaling pathways drive an output neuron population configured to generate motor output.

## Claims

### Claim 1

A neural network, comprising:
multiple modalities, wherein each modality comprises multiple neurons; and
an interconnection lattice for cross-associating signaling between the neurons in the different modalities.

### Claim 2 (depends on 1)

The neural network of claim 1, wherein the interconnection lattice comprises:
a plurality of reciprocal signaling pathways for directed information flow between different
modalities, said plurality of reciprocal signaling pathways comprising top-down signaling pathways and bottom-up signaling pathways configured for information flow in a first direction and a second direction opposite to the first direction, respectively;
wherein each bottom-up signaling pathway has a reciprocal top-down signaling pathway, such that bottom-up signaling pathways for a first set of modalities influence top-down signaling pathways for a second set of modalities via learning rules.

### Claim 3 (depends on 2)

The neural network of claim 2, wherein:
each neuron generates a signal in response to input signals from one or more other neurons via the interconnection lattice; and
each modality further includes a perception neuron population and an action neuron population, such that each perception neuron has a corresponding action neuron.

### Claim 4 (depends on 3)

The neural network of claim 3, wherein:
a perception neuron population at an input periphery of the neural network is designated as an input neuron population configured to receive sensory input; and
an action neuron population at an output periphery of the neural network is designated as an output neuron population configured to generate motor output.

### Claim 5 (depends on 4)

The neural network of claim 4, wherein:
the interconnection lattice further includes a plurality of perception neuron populations along a number of bottom-up signaling pathways, and a plurality of action neuron populations along a number of top-down signaling pathways;
wherein the input neuron population drives perception neurons along a number of bottom-up
signaling pathways;
wherein a first set of perception neurons along bottom-up signaling pathways drive a first set of action neurons along top-down signaling pathways; and
wherein action neurons along a number of top-down signaling pathways drive the output neuron population.

### Claim 6 (depends on 5)

The neural network of claim 5, wherein:
each perception neuron along a bottom-up signaling pathway is trained using a learning rule based on the firing events of said perception neuron and the firing events of the corresponding action neuron along a reciprocal top-down signaling pathway; and
each action neuron along a top-down signaling pathway is trained using a learning rule based on the firing events of said action neuron and the firing events of the corresponding perception neuron along a reciprocal bottom-up signaling pathway.

### Claim 7 (depends on 6)

The neural network of claim 6, wherein the bottom-up signaling pathways are arranged in an acyclic bottom-up digraph.

### Claim 8 (depends on 7)

The neural network of claim 7, wherein the top-down signaling pathways are arranged in an acyclic top-down digraph, such that each top-down signaling pathway in the top-down digraph has a reciprocal bottom-up signaling pathway in the bottom-up digraph, and each action neuron population along a top-down signaling pathway in the top-down digraph corresponds to a perception neuron population along a reciprocal bottom-down pathway in the bottom-up digraph.

### Claim 9

A neural network, comprising:
a first set of neural nodes, wherein each node comprises multiple neuron populations including multiple neurons;
a second set of neural nodes, wherein each node of the second set is a union of at least two nodes of the first set; and
an interconnect network comprising multiple directed edges that connect neuron in nodes of the first set with neurons in nodes of the second set, such that a connected node of the second set exchanges signals with at least two nodes of the first set via the interconnect network;
wherein nodes of the first and second set are arranged in a lattice; and
wherein each neuron generates a firing signal in response to input signals from one or more other neurons via the interconnect network.

### Claim 10 (depends on 9)

The neural network of claim 9, wherein neuron populations in a node are interconnected via multiple directed edges arranged in an acyclic digraph, each edge comprising a signaling pathway in the interconnect network.

### Claim 11 (depends on 10)

The neural network of claim 10, wherein the interconnect network interconnects said nodes via bottom-up signaling pathways arranged in an acyclic bottom-up digraph in the interconnect network, each bottom-up signaling pathway including one or more neuron populations and directed edges.

### Claim 12 (depends on 11)

The neural network of claim 11, wherein a first neuron population in a first node is interconnected to a second neuron population in a second node only if the second node is a superset of the first node.

### Claim 13 (depends on 12)

The neural network of claim 12, wherein the interconnect network further interconnects said nodes via top-down signaling pathways arranged in an acyclic top-down digraph in the interconnect network, each top-down signaling pathway including one or more neuron populations and directed edges.

### Claim 14 (depends on 13)

The neural network of claim 13, wherein:
each neuron population in the top-down digraph corresponds to a neuron population in the bottom-up digraph; and
each top-down signaling pathway in the top-down digraph has a reciprocal bottom-up signaling pathway in the bottom-up digraph, wherein information flows along said top-down signaling pathway in a first direction, and information flows along said reciprocal bottom-up signaling pathway in a direction opposite of the first direction.

### Claim 15 (depends on 14)

The neural network of claim 14, wherein:
a neuron population at an input periphery of the neural network is designated as an input neuron population configured to receive sensory input, wherein the input neuron population drives neurons along a number of bottom-up signaling pathways;
a first set of neurons along bottom-up signaling pathways drive a first set of neurons along top-down signaling pathways; and
a neuron population at an output periphery of the neural network is designated as an output neuron population configured to generate motor output, wherein neurons along a number of top-down signaling pathways drive the output neuron population.

### Claim 16 (depends on 15)

The neural network of claim 15, wherein:
each neuron along a bottom-up signaling pathway is trained using a learning rule based on
the firing events of said neuron and the firing events of the corresponding neuron along a reciprocal top-down signaling pathway; and
each neuron along a top-down signaling pathway is trained using a learning rule based on the firing events of said neuron and the firing events of the corresponding neuron along a reciprocal bottom-up signaling pathway.

### Claim at position 17

17-23. (canceled)

### Claim 24

A computer program product on a computer-readable medium for cross-associating signaling in a neural network comprising a plurality of neural nodes connected via an interconnect network comprising bottom-up signaling pathways and top-down signaling pathways arranged in a lattice, wherein each node has a sensory-motor modality and generates a signal in response to input signals received from one or more other nodes via the interconnect network, the computer program product comprising instructions which when executed on a computer cause the computer to perform operations including:
connecting a plurality of first nodes in a first set of nodes with a plurality of second nodes in a second set of nodes, such that a connected node in the second set exchanges signals with at least two nodes in the first set via the interconnect network;
for a node comprising one or more neuron populations, interconnecting the neuron populations within said node via multiple directed edges arranged in an acyclic digraph, wherein each neuron population comprises one or more neurons, and each edge includes a signaling pathway in the interconnect network;
interconnecting said plurality of neural nodes via bottom-up signaling pathways arranged in an acyclic bottom-up digraph in the interconnect network, each bottom-up signaling pathway including one or more neuron populations and directed edges;
interconnecting said plurality of neural nodes via top-down signaling pathways arranged in an acyclic top-down digraph in the interconnect network, each top-down signaling pathway including one or more neuron populations and directed edges, wherein each neuron population in the top-down digraph corresponds to a neuron population in the bottom-up digraph, wherein each top-down signaling pathway in the top-down digraph has a reciprocal bottom-up signaling pathway in the bottom-up digraph, such that information flows along said top-down signaling pathway in a first direction, and information flows along said reciprocal bottom-up signaling pathway in a direction opposite of the first direction;
designating a neuron population at an input periphery of the neural network as an input neuron population configured to receive sensory input, wherein the input neuron population drives neurons along a number of bottom-up signaling pathways;
a first set of neurons along bottom-up signaling pathways driving a first set of neurons along top-down signaling pathways;
designating a neuron population at an output periphery of the neural network as an output neuron population configured to generate motor output, wherein neurons along a number of top-down signaling pathways drive the output neuron population;
training each neuron along a bottom-up signaling pathway using a learning rule based on the firing events of said neuron and the firing events of the corresponding neuron along a reciprocal top-down signaling pathway;
training each neuron along a top-down signaling pathway using a learning rule based on the firing events of said neuron and the firing events of the corresponding neuron along a reciprocal bottom-up signaling pathway; and
interconnecting neuron populations in the bottom-up digraph to neuron populations in the top-down digraph using additional directed edges.
