# US 2019/0318242 A1 · Learning algorithms for oscillatory memristive neuromorphic circuits > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318242 A1 - Document ID: US20190318242A1 - Status: Published application - Kind: A1 - Application number: 16/345,551 - Filed: 2017-10-27 - Published: 2019-10-17 - Earliest claimed: 2016-10-27 - Assignees: University of Florida Research Foundation, Inc. - Applicants: University of Florida Research Foundation, Inc. - Inventors: Jack D. KENDALL; Juan C. NINO - Web page: https://patentgrep.com/patent/US20190318242A1 ## CPC - G06N 3/08: Learning methods - G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/045: Combinations of networks - G06N 3/0455: Auto-encoder networks; Encoder-decoder networks - G06N 3/0475: Generative networks - G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs - G06N 3/0499: Feedforward networks - G06N 3/063: using electronic means - G06N 3/065: Analogue means - G06N 3/09: Supervised learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Learning algorithms for oscillatory memristive neuromorphic circuits are described. In one embodiment, a neuromorphic circuit learning network includes a number of neuromorphic circuit nodes, each including a recognition neuron unit and a generative neuron unit. The learning network further includes a plurality of neuromorphic circuit feedforward couplings between the recognition neuron units in the neuromorphic circuit nodes, and a plurality of neuromorphic circuit feedback couplings between the generative neuron units in the neuromorphic circuit nodes. The learning network also includes a learning controller configured to drive activity among the recognition neuron units and train the generative neuron units for learning in one mode and to drive activity among the generative neuron units and train the recognition neuron units for learning in another mode. Various deep learning algorithms can be implemented in the learning network. Two examples include the wake-sleep algorithm for unsupervised neural networks and target propagation. ## Claims ### Claim 1 A neuromorphic circuit learning network, comprising: a plurality of neuromorphic circuit nodes, each neuromorphic circuit node including a recognition neuron unit and a generative neuron unit; a plurality of neuromorphic circuit feedforward couplings between the recognition neuron units in the neuromorphic circuit nodes; a plurality of neuromorphic circuit feedback couplings between the generative neuron units in the neuromorphic circuit nodes; and a learning controller configured, in a first mode, to: drive activity among the recognition neuron units through the neuromorphic circuit feedforward couplings; and train the generative neuron units for learning through the neuromorphic circuit feedback couplings. ### Claim 2 (depends on 1) The neuromorphic circuit learning network according to claim 1, wherein the learning controller is further configured, in the first mode, to train the generative neuron units for learning during a second mode based on the activity. ### Claim 3 (depends on 2) The neuromorphic circuit learning network according to claim 2, wherein the learning controller is further configured, in a second mode, to: drive activity among the generative neuron units through the neuromorphic circuit feedback couplings; and train the recognition neuron units for learning through the neuromorphic circuit feedforward couplings. ### Claim 4 (depends on 3) The neuromorphic circuit learning network according to claim 3, wherein the learning controller is further configured, in the second mode, to train the recognition neuron units for learning during the first mode based on the activity of the generative neuron units. ### Claim 5 (depends on 1) The neuromorphic circuit learning network according to claim 1, wherein: the plurality of neuromorphic circuit nodes form a number of layers in the neuromorphic circuit learning network; and in the first mode, a training example is shown to a lowest layer in the neuromorphic circuit learning network. ### Claim 6 (depends on 2) The neuromorphic circuit learning network according to claim 2, wherein: the plurality of neuromorphic circuit nodes form a number of layers in the neuromorphic circuit learning network; and in a second mode, a random vector is shown to a highest layer in the neuromorphic circuit learning network. ### Claim 7 (depends on 2) The neuromorphic circuit learning network according to claim 2, wherein the learning controller is further configured to train feedback weights for operation as an autoencoder. ### Claim 8 (depends on 1) The neuromorphic circuit learning network according to claim 1, wherein at least one of the plurality of neuromorphic circuit feedforward couplings or at least one of the plurality of neuromorphic circuit feedforward couplings comprises at least one memristor. ### Claim 9 (depends on 1) The neuromorphic circuit learning network according to claim 1, wherein at least one of the plurality of neuromorphic circuit feedforward couplings or at least one of the plurality of neuromorphic circuit feedforward couplings comprises: a long-term potentiation (LTP) memristor arranged in parallel with a long-term depression (LTD) memristor; and a difference amplifier that generates a difference signal based on a difference between output signals from the LTP memristor and the LTD memristor. ### Claim 10 A neuromorphic circuit learning network, comprising: a plurality of neuromorphic circuit nodes, each neuromorphic circuit node including a recognition neuron unit and a generative neuron unit; a plurality of neuromorphic circuit feedforward couplings between the recognition neuron units in the neuromorphic circuit nodes; a plurality of neuromorphic circuit feedback couplings between the generative neuron units in the neuromorphic circuit nodes; and a learning controller configured to drive activity among the recognition neuron units and train the generative neuron units for learning. ### Claim 11 (depends on 10) The neuromorphic circuit learning network according to claim 10, wherein the learning controller is further configured to drive activity among the generative neuron units and train the recognition neuron units for learning. ### Claim 12 (depends on 10) The neuromorphic circuit learning network according to claim 10, wherein the learning controller is further configured, in a first mode, to train the generative neuron units for learning through the neuromorphic circuit feedback couplings during a second mode. ### Claim 13 (depends on 10) The neuromorphic circuit learning network according to claim 10, wherein the learning controller is further configured, in a second mode, to train the recognition neuron units for learning through the neuromorphic circuit feedforward couplings during a first mode. ### Claim 14 (depends on 10) The neuromorphic circuit learning network according to claim 10, wherein at least one of the plurality of neuromorphic circuit feedforward couplings or at least one of the plurality of neuromorphic circuit feedforward couplings comprises at least one memristor. ### Claim 15 (depends on 10) The neuromorphic circuit learning network according to claim 10, wherein at least one of the plurality of neuromorphic circuit feedforward couplings or at least one of the plurality of neuromorphic circuit feedforward couplings comprises: a long-term potentiation (LTP) memristor arranged in parallel with a long-term depression (LTD) memristor; and a difference amplifier that generates a difference signal based on a difference between output signals from the LTP memristor and the LTD memristor. ### Claim 16 A method of training a neuromorphic circuit learning network, comprising, in a first mode: driving activity among recognition neuron units in a plurality of neuromorphic circuit nodes through feedforward couplings in the plurality of neuromorphic circuit nodes; training generative neuron units in the plurality of neuromorphic circuit nodes for learning through feedback couplings in the plurality of neuromorphic circuit nodes. ### Claim 17 (depends on 16) The method of training a neuromorphic circuit learning network according to claim 16, wherein the training comprises training the generative neuron units for learning during a second mode based on the activity. ### Claim 18 (depends on 16) The method of training a neuromorphic circuit learning network according to claim 16, further comprising, in a second mode: driving activity among the generative neuron units in the plurality of neuromorphic circuit through the feedback couplings; and training the recognition neuron units for learning through the feedforward couplings. ### Claim 19 (depends on 16) The method of training a neuromorphic circuit learning network according to claim 16, wherein: the plurality of neuromorphic circuit nodes form a number of layers in the neuromorphic circuit learning network; and the method further comprises showing a training example to a lowest layer in the neuromorphic circuit learning network in the first mode. ### Claim 20 (depends on 18) The method of training a neuromorphic circuit learning network according to claim 18, wherein: the plurality of neuromorphic circuit nodes form a number of layers in the neuromorphic circuit learning network; and the method further comprises showing a random vector to a highest layer in the neuromorphic circuit learning network in the second mode.