Recurrent neural network architectures based on synaptic connectivity graphs

A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

• obtaining an input sequence comprising an input element at each of a plurality of input positions;
• and
• processing the input sequence using a recurrent neural network to generate a network output, wherein the recurrent neural network comprises a brain emulation subnetwork having a network architecture that has been determined according to a synaptic connectivity graph, wherein the synaptic connectivity graph represents synaptic connectivity between neurons in a brain of a biological organism, the processing comprising:
• at a first time step, processing a first input element in the input sequence to generate a hidden state of the recurrent neural network;
• at each of a plurality of subsequent time steps, updating the hidden state of the recurrent neural network based on i) a subsequent input element in the input sequence and ii) a current value of the hidden state;
• and
• at each of one or more of the plurality of time steps, generating an output element for the time step based on the updated hidden state for the time step.

One or more non-transitory storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

• obtaining an input sequence comprising an input element at each of a plurality of input positions;
• and
• processing the input sequence using a recurrent neural network to generate a network output, wherein the recurrent neural network comprises a brain emulation subnetwork having a network architecture that has been determined according to a synaptic connectivity graph, wherein the synaptic connectivity graph represents synaptic connectivity between neurons in a brain of a biological organism, the processing comprising:
• at a first time step, processing a first input element in the input sequence to generate a hidden state of the recurrent neural network;
• at each of a plurality of subsequent time steps, updating the hidden state of the recurrent neural network based on i) a subsequent input element in the input sequence and ii) a current value of the hidden state;
• and
• at each of one or more of the plurality of time steps, generating an output element for the time step based on the updated hidden state for the time step.