# US 2014/0156574 A1 · Rate stabilization through plasticity in spiking neuron network > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0156574 A1 - Document ID: US20140156574A1 - Status: Published application - Kind: A1 - Application number: 13/691,554 - Filed: 2012-11-30 - Published: 2014-06-05 - Earliest claimed: 2012-11-30 - Assignees: Brain Corporation - Applicants: Brain Corporation - Inventors: Filip Piekniewski; Micah Richert; Dimitry Fisher; Eugene Izhikevich - Web page: https://patentgrep.com/patent/US20140156574A1 ## CPC - G06N 3/02: Neural networks - G06N 3/049: Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Apparatus and methods for activity based plasticity in a spiking neuron network adapted to process sensory input. In one embodiment, the plasticity mechanism may be configured for example based on activity of one or more neurons providing feed-forward stimulus and activity of one or more neurons providing inhibitory feedback. When an inhibitory neuron generates an output, inhibitory connections may be potentiated. When an inhibitory neuron receives inhibitory input, the inhibitory connection may be depressed. When the inhibitory input arrives subsequent to the neuron response, the inhibitory connection may be depressed. When input features are unevenly distributed in occurrence, the plasticity mechanism is capable of reducing response rate of neurons that develop receptive fields to more prevalent features. Such functionality may provide network output such that rarely occurring features are not drowned out by more widespread stimulus. ## Claims ### Claim 1 An apparatus configured to process input signals in spiking network, the network comprising an excitatory and an inhibitory neuron, the apparatus comprising: a storage apparatus configured to store said spiking network; logic, in data communication with said storage apparatus, said logic configured to: based on an excitatory response by the excitatory neuron: strengthen a connection communicating the excitatory response to the inhibitory neuron; and strengthen a connection communicating an inhibitory response from the inhibitory neuron; and based on the inhibitory response by the inhibitory neuron: weaken the connection communicating the excitatory response to the inhibitory neuron; and weaken the connection communicating the inhibitory response from the inhibitory neuron. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein: the excitatory response is based on at least a portion of the input signal; and the excitatory response is configured to increase a probability of response being generated by the inhibitory neuron. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein: the inhibitory response is based at least in part on the excitatory response; and the inhibitory response is configured to reduce a probability of response being generated by the excitatory neuron. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein: the connection communicating the excitatory response is characterized by an excitatory efficacy; and the strengthening of the connection communicating the excitatory response comprises an increase in the excitatory efficacy based at least on the excitatory response being generated by the excitatory neuron. ### Claim 5 (depends on 4) The apparatus of claim 4, wherein: the connection communicating the inhibitory response is characterized by an inhibitory efficacy; and the strengthening of the connection communicating an inhibitory response comprises an increase in the inhibitory efficacy based at least on the excitatory response being generated by the excitatory neuron. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein: the connection communicating the inhibitory response is characterized by an inhibitory efficacy; and the weakening of the connection communicating the inhibitory response comprises a decrease in the inhibitory efficacy based at least on the inhibitory response being generated by the inhibitory neuron. ### Claim 7 (depends on 5) The apparatus of claim 5, wherein: the connection communicating the excitatory response is characterized by an excitatory efficacy; and the weakening of the connection communicating the excitatory response is characterized by a decreasing in the excitatory efficacy based at least on the inhibitory response being generated by the inhibitory neuron. ### Claim 8 A non-transitory computer-readable storage medium having instructions resident thereon, the instructions being executable by one or more processors to implement activity-dependent plasticity in a spiking neuron network signal processing apparatus, said instructions configured to, when executed: operate an excitatory network portion configured to generate an excitatory output based at least on the signal; operate an inhibitory network portion configured to generate an inhibitory output based at least on the excitatory output, the excitatory output being characterized by an excitatory firing rate; for a given value of a parameter associated with the signal, adjust an efficacy of one or more connections associated with the inhibitory output so as to substantially equalize the excitatory firing rate of individual neurons of the excitatory network portion. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein: the excitatory output and the inhibitory output are characterized by excitatory and inhibitory output rates, respectively; and the parameter comprises a ratio of the excitatory output rate to the inhibitory output rate. ### Claim 10 (depends on 8) The apparatus of claim 8, wherein: individual ones of the one or more connections are configured to provide the excitatory output to individual neurons of the inhibitory network portion; the excitatory firing rate of individual neurons of the excitatory network portion is characterized by a firing rate range; and the equalization of the excitatory firing rate is characterized by a smaller value of the firing rate range compared to value of the firing rate range in absence of the plasticity adjustment. ### Claim 11 A computerized spiking neuron apparatus comprising one or more processors configured to execute computer program modules, wherein the computer program modules are configured to, when executed, cause the one or more processors to: based on generating an output, increase an efficacy of one or more inhibitory connections, the one or more connections configured to provide an inhibitory input to a neuron; and based on receiving the inhibitory input, reduce the efficacy. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein: the inhibitory input comprises am output of one or more inhibitory neurons; and the increase in the efficacy and reduction in the efficacy cooperate to produce a negative feedback loop between the neuron and individual ones of the one or more inhibitory neurons. ### Claim 13 (depends on 12) The apparatus of claim 12, wherein: individual ones the one or more connections are associated with individual ones of the one or more inhibitory neurons; and the increasing the efficacy and reducing the efficacy are configured to modify efficacy of individual ones of the one or more inhibitory connections. ### Claim 14 (depends on 12) The apparatus of claim 12, wherein: the one or more inhibitory connections comprise a strong connection having a first efficacy associated therewith and a weak connection having a second efficacy associated therewith; and when the second efficacy being smaller than the first efficacy, the increasing the efficacy and reducing the efficacy cooperate to increase the second efficacy and to reduce the first efficacy. ### Claim 15 (depends on 14) The apparatus of claim 14, wherein the first efficacy is characterized by first weight and the second efficacy is characterized by second weight, the first weight being greater than the second weight. ### Claim 16 (depends on 14) The apparatus of claim 14, wherein the first efficacy is characterized by first latency and the second efficacy is characterized by second latency, the first latency being smaller than the second latency. ### Claim 17 (depends on 11) The apparatus of claim 11, wherein: individual ones of the of one or more inhibitory connections are characterized by an activity-based plasticity mechanism comprising: a pre-synaptic rule configured based on at least the receiving the inhibitory stimulus prior to the generating the output; and a post-synaptic rule configured based on at least the generating the output prior to the receiving the inhibitory stimulus; and adjustment of the efficacy due to the post-synaptic rule is configured to be negatively-correlated with adjustment of the efficacy due to the pre-synaptic rule. ### Claim 18 (depends on 17) The apparatus of claim 17, wherein the negatively-correlated plasticity adjustment configuration is characterized by: the pre-synaptic rule being configured to cause negative efficacy change; and the post-synaptic rule being configured to cause positive efficacy change within at least a time window subsequent to the receipt of the inhibitory stimulus. ### Claim 19 (depends on 18) The apparatus of claim 18, wherein: the apparatus is configured to process sensory input refreshed at least at a 40 ms interval; and the time window is selected from the range between 5 and 20 ms inclusive. ### Claim 20 (depends on 17) The apparatus of claim 17, wherein the negatively-correlated plasticity adjustment configuration is characterized by: the pre-synaptic rule being configured to cause negative efficacy change; the post-synaptic rule being configured to cause: a negative efficacy change within at least a time window subsequent to the receipt of the inhibitory stimulus; and a positive efficacy change subsequent to the time window. ### Claim 21 A computerized method of processing input stimulus by a spiking neuron network comprising first selectivity and second selectivity neurons, the method comprising: causing the first selectivity neuron to respond to first feature of the input stimulus, the first selectivity neuron response being characterized by first rate configured based on a frequency of occurrence of the first feature; causing the second selectivity neuron to respond to second feature of the input stimulus, the second selectivity neuron response being characterized by second rate configured based on a frequency of occurrence of the second feature; and if the frequency of occurrence of the first feature is greater than the frequency of occurrence of the second feature, applying a rate reduction mechanism configured to reduce the first rate based on one or more response of the second selectivity neurons. ### Claim 22 (depends on 21) The method of claim 21, wherein: the rate reduction is configured to compress a spike output comprising the first selectivity neuron responses; and the compression is characterized by a compression parameter configured based on the frequency of occurrence of the first feature. ### Claim 23 (depends on 21) The method of claim 21, wherein: the first neuron is characterized by a first receptive field having at least a first dimension associated therewith; the second neuron is characterized by a second receptive field having at least a second dimension associated therewith; and when the frequency of occurrence of the first feature is greater than the frequency of occurrence of the second feature, the at least second dimension is greater than the at least first dimension. ### Claim 24 (depends on 21) The method of claim 21, further comprising: causing a first network portion to respond to the first feature, the first network portion comprising the first neuron; and causing a second network portion to respond to the second feature, the second network portion comprising the second neuron; wherein the rate equalization mechanism is configured to, when the frequency of occurrence of the first feature is greater than the frequency of occurrence of the second feature, reduce number of output spikes generated by the first portion within a time interval relative to a number of output spikes generated by the first portion within the time interval absent the rate equalization mechanism. ### Claim 25 (depends on 21) The method of claim 21, wherein the method is configured to cause a ratio of cumulative number of spikes generated by the first neuron based on the first feature to substantially equal to cumulative number of spikes generated by the second neuron based on the second feature to be in the range between 0.5 and 2, inclusive.