Apparatus and method for enhancing transfer function non-linearities in pulse frequency encoded neurons
AssigneesRicoh Company, Ltd., Ricaoh Corporation
InventorsDavid Geoffrey Stork, Ronald Craig Keesing
Life of this patent
Estimating the term…
Abstract
A method and apparatus is presented for synthesizing a network for use with pulse frequency encoded signals that has a smoothly saturating transfer characteristic for large signals based on the use of delay and an OR-gate. When connected to the output of a pulse frequency type of neuron, it results in a sigmoidal activation function.
Claims11
A network for operating on a pulse frequency encoded input signal for producing a pulse frequency encoded output signal with a modified pulse rate, the network comprising:
- a) an input terminal for inputting the pulse frequency encoded input signal;
- b) a delay circuit with an input connected to the input terminal for inputting the pulse frequency encoded input signal and outputting, at an output, a delayed pulse frequency encoded input signal, delayed by at least one minimal pulse interval;
- and
- c) an OR-gate with an output, a first input connected to the input terminal, and a second input connected to the output of the delay circuit, for OR-ing the pulse frequency encoded input signal with the delayed pulse frequency input signal, and for producing a pulse frequency encoded output signal at the output with an increased pulse rate relative to the pulse frequency encoded input signal pulse rate, the increased pulse rate increasing monotonically relative to the pulse frequency encoded input signal pulse rate and smoothly approaching a saturation rate equal to a prescribed maximum allowable pulse frequency encoded signal pulse rate.
An artificial pulse frequency type neuron with improved pulse frequency encoded sigmoidal squashing characteristics comprising:
- a) a pulse frequency type neuron with a prescribed maximum pulse operating rate, the pulse frequency type neuron including,
- i) an excitatory vector input terminal for inputting a pulse frequency encoded excitatory vector input signal with at least one vector element,
- ii) an inhibitory vector input terminal for inputting a pulse frequency encoded inhibitory vector input signal with at least one vector element,
- iii) a synaptic weighting vector input terminal for inputting a pulse frequency encoded synaptic weighting vector, and
- iv) an output terminal or outputting a pulse frequency encoded neuron output signal that is a combined excitatory and inhibitory pulse frequency encoded synaptic weighted signal;
- b) a sigmoidal enhancement circuit having an input pulse rate to output pulse rate transfer nonlinear characteristic that monotonically increases the output pulse rate, relative to the input pulse rate, the output rate smoothly approaching a prescribed maximum allowable rate corresponding to the pulse frequency encoded neuron prescribed maximum pulse operating rate, for operating on the pulse frequency encoded neuron output signal, a for producing at an output a pulse frequency encoded signal with a modified pulse rate, the sigmoidal enhancement circuit comprising,
- i) an enhancement circuit input terminal connected to the output of the pulse frequency type neuron,
- ii) delay circuit with an input connected to the output terminal of the pulse frequency type neuron for inputting the pulse frequency encoded neuron output signal and outputting a delayed pulse frequency encoded neuron output signal, delayed by at least one nominal pulse interval, and
- iii) OR-gate means having a first input terminal connected to the enhancement circuit input terminal, a second input terminal connected to the output of the delay means for producing the pulse frequency encoded output signal with a modified pulse rate.
A method for transforming a pulse frequency encoded input signal and directly producing, without decoding, a pulse frequency encoded output signal with a modified pulse rate, the method steps comprising:
- a) inputting a pulse frequency encoded input signal;
- b) delaying the pulse frequency encoded input signal by at least one nominal pulse interval;
- and
- c) logically OR-ing the pulse frequency encoded input signal and the delayed pulse frequency encoded input signal and producing a pulse frequency encoded output signal with a modified pulse rate, the modified pulse rate being a nonlinear function of the pulse frequency encoded input signal rate that increases monotonically with the pulse frequency encoded input signal rate and smoothly approaches a maximum pulse rate corresponding to a prescribed maximum pulse frequency encoded signal pulse rate.
Dependencies are inferred from the claim text.