Legendre memory units in recurrent neural networks

A method comprising:

• defining, by a computer processor, a node response function for each node in a network, the node response function representing a state over time, wherein the state is encoded into one of binary events or real values, each node having a node input and a node output;
• defining, by the computer processor, a set of connection weights with each node input;
• defining, by the computer processor, a set of connection weights with each node output;
• defining, by the computer processor, one or more Legendre Memory Unit (LMU) cells having a set of recurrent connections defined as a matrix that determines node connection weights based on the formula:
• A=[a]ij∈q×q
• where
• ⁢
• ⁢
• a
• ij
• =
• (
• 2
• ⁢
• i
• +
• 1
• )
• ⁢
• {
• -
• 1
• i
• <
• j
• (
• -
• 1
• )
• i
• -
• j
• +
• 1
• i
• ≥
• j
• where q is an integer determined by the user, and i and j are greater than or equal to zero;
• and
• generating, by the computer processor, a recurrent neural network comprising the node response function for each node, the set of connection weights with each node input, the set of connection weights with each node output, and the LMU cells by training the network as a recurrent neural network by updating a plurality of its parameters or by fixing one or more parameters while updating the remaining parameters.