US 5,333,239 AGranted

Learning process system for use with a neural network structure data processing apparatus

AssigneeFujitsu Limited

InventorsNobuo Watanabe, Takashi Kimoto, Akira Kawamura, Ryusuke Masuoka, Kazuo Asakawa

Life of this patent

Estimating the term…

Loading patent…

Abstract

A learning process system is provided for a neural network. The neural network is a layered network comprising an input layer, an intermediate layer and an output layer formed of basic units. In the basic units, a plurality of inputs is multiplied by a weight signal and the products are accumulated, thereby supplying the sum of products. An output signal is obtained using a threshold value function in response to the sum of products. An error signal is generated by an error circuit in response to a difference between the output signal obtained from the output layer and a teacher signal. A weight updating signal is determined in a weight learning circuit by obtaining a weight value in which the sum of the error values falls within an allowable range. Thus, the learning is performed in the layered neural network through use of a back propagation method. Through such learning in the layered neural network, an updating quantity to be obtained in the present weight updating cycle is determined in response to a once delayed weight updating quantity signal in a previous weight updating cycle and a twice delayed weight updating quantity obtained at a twice-previous weight updating cycle prior to the previous weight updating cycle.

Claims23

Dependent claims

A learning process system for use with a neural network comprising an input layer including a first plurality of basic units, each including a basic unit input each connected to receive an input signal and a basic unit output each connected to receive a weight signal to be multiplied by the input signal received by the basic unit input, connected to provide a sum-of-products and connected to convert the sum-of-products by a threshold value function to provide a final result signal at a basic unit output, at least one intermediate layer including a second plurality of basic units, each including a basic unit input each connected to receive another input signal and a basic unit output each connected to receive the weight signal to be multiplied by the another input signal received by the basic unit input, connected to provide another sum-of-products and connected to convert the another sum-of-products by another threshold value function to provide another final result signal at the basic unit output, an output layer including at least one basic unit, each including a basic unit input each connected to receive a further input signal and a basic unit output each connected to receive the weight signal to be multiplied by the further input signal received by the basic unit input, connected to provide a further sum-of-products and connected to convert the further sum-of-products by a further threshold value function to provide a further final result signal at the basic unit output, and internal connections from the basic unit outputs of the input layer to the basic unit inputs of the at least one intermediate layer and from the basic unit outputs of the at least one intermediate layer to the basic unit inputs of the output layer, said learning process system comprising:

output signal providing means for, when a plurality of predetermined input signals are supplied to the input layer basic unit inputs, receiving first output signals from the basic unit output of each at least one basic unit of the output layer and for providing second output signals identical to the first output signals,
error value means for receiving the second output signals and a teacher signal and for generating error signals designating an inconsistency between the second output signal and the teacher signal, and
weight learning means for calculating and storing a weight updating signal in respective updating cycles and for generating the weight signal so that a sum of the error signals becomes within a predetermined allowable range by sequentially updating in updating cycles the weight signal in accordance with the weight updating signal based on the sum of the error signals determined at a first updating cycle, based on a once dalyed weight updating signal determined at a second updating cycle prior to the first updating cycle, and based on a twice delayed weight updating signal determined at a third updating cycle prior to the second updating cycle.

A learning process system for use in a network, comprising:

a single layer including a plurality of basic units, each including a basic unit input each connected to receive an input signal and a basic unit output each connected to receive a weight signal to be multiplied by the input signal received by the basic unit input, connected to provide a sum-of-products and connected to convert the sum-of-products by a threshold value function to provide an output signal at a basic unit output;
a feedback path for selectively feeding back output signals at basic unit outputs as inputs to basic unit inputs to sequentially operate as an input layer, an intermediate layer and an output layer;
output signal providing means for receiving a first output signal from the basic unit output of at least one basic unit of said single layer when operating as the output layer and providing a second output signal based on the first output signal by supplying a plurality of predetermined input signals to the basic unit input of at least one basic unit of said single layer when operating as the input layer;
error value means for obtaining the second output signal from said output signal providing means and a teacher signal and for calculating an error signal designating an inconsistency between the second output signal and the teacher signal;
and
weight learning means for calculating and storing a weight updating signal in respective updating cycles and for producing the weight signal so that a sum of the error signals becomes within a predetermined allowable range by sequentially updating in updating cycles the weight signal in accordance with the weight updating signal based on the sum of the error signals determined at a first one of the updating cycles, based on a once delayed weight updating signal determined at a second updating cycle prior to the first updating cycle and based on a twice delayed weight updating signal determined at a third updating cycle prior to the second updating cycle.

3 more elements

Dependencies are inferred from the claim text.

Facts

Application: 08/003,856Filed 11 Jan 1993 · continuation
Priority: JP 63-22782512 Sep 1988
Granted: 26 Jul 1994
Term ends: Not estimatedThe source records don’t support an estimate.

Classification6 codes

G06N 3/084
Learning methods: backpropagation, e.g. using gradient descent
G06N 3/04
Neural networks: architecture, e.g. interconnection topology
G06N 3/0442
Recurrent networks, e.g. Hopfield networks: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
G06N 3/0499
Architecture, e.g. interconnection topology: feedforward networks