Method of assigning initial values of connection parameters to a multilayered neural network
OwnerToru Nikiinventor
Life of this patent
Estimating the term…
Abstract
A generator in a back propagation type neural network having an input layer, an output layer and an intermediate layer coupled the input and output layers forms initial values for connection parameters. A first generator produces an initial value W10 of a weight coefficient of each connection parameter of the intermediate layer from in-class covariant data SW and inter-class co-variant data SB over data inputted to the input layer. The produced values are set into respective synapses of the intermediate layer as connection parameters.
Claims22
An apparatus for classifying input data representing a physical quantity into predetermined k classes, each class designating a set including a plurality of data which have similar characteristics, said apparatus comprising
- a neural network comprising an input layer, an output layer, and an intermediate layer coupled between the input layer and the output layer, and
- determining means for determining connection parameters between the layers of said neural network by using training data prior to classifying the input data, the connection parameters being represented at least by a weight coefficient,
- wherein the input layer includes means for inputting data;
- the intermediate layer includes means for receiving data from the input layer and processing the received data by means of the connection parameters;
- the output layer includes means for receiving data processed by the intermediate layer, processing the received data by means of the connection parameters, and outputting signals indicating one of the predetermined k classes;
- and
- the determining means comprises:
- a first generating means for generating initial value data W.sub.10 of the weight coefficient of each connection parameter of the intermediate layer from in-class dispersion data S.sub.W and inter-class dispersion data S.sub.B of the training data over all classes into which the training data has been partitioned, said in-class dispersion data S.sub.W representing a dispersion degree of the training data within a class and said inter-class dispersion data S.sub.B representing a dispersion degree of the training data over classes;
- setting means for setting the generated initial value data into the respective synapses of the intermediate layer, as connection parameters;
- and
- correction means for correcting the connection parameters by using a back-propagation learning method.
A method for classifying input data representing a physical quantity into predetermined k classes, each class designating a set including a plurality of data which have a similar characteristic, by using a back propagation type neural network having an input layer, an output layer and an intermediate layer coupled between the input and output layers, each layer comprising a network of synapse elements and connections between the synapse elements, each connection being represented by a connection parameter which includes at least a weight coefficient, the method comprising the steps of:
- (a) partitioning a plurality of training data x into k classes;
- (b) generating data representing an in-class dispersion matrix S.sub.W and data representing an inter-class dispersion matrix S.sub.B of the training data over all classes into which the training data has been partitioned, each class designating a set in which a plurality of training data which have similar characteristics are included, said in-class dispersion data S.sub.W representing a dispersion degree of the training data within a class and said inter-class dispersion data S.sub.B representing a dispersion degree of the training data over classes on the basis of the plurality of input data;
- (c) determining initial values W.sub.10 of the weight coefficients of the synapses of the intermediate layer on the basis of the data representing said inter-class dispersion matrix S.sub.W and said in-class dispersion matrix S.sub.B ;
- and
- (d) determining final values of said weight coefficients of the synapses of the intermediate layer beginning with the initial values W.sub.10 according to back propagation type learning method in which training is input via said input layer, and setting connection parameters including the determined final weight coefficients into the respective synapses of the intermediate layer;
- (e) inputting input data to the input layer;
- (f) processing the input data through the intermediate layer having the determined weight coefficients;
- and
- (g) further processing the data processed through the intermediate layer by means of the output layer, and outputting from the output layer signals indicating one of the predetermined k classes to which the input data belongs.
An apparatus for classifying input data representing a physical quantity into predetermined k classes, each class designating a set including a plurality of data which have similar characteristics, said apparatus comprising:
- a back propagation type neural network including an input layer, an output layer, and an intermediate layer coupled between the input layer and the output layer;
- and
- determining means for determining connection parameters between the layers of said neural network by using training data prior to classifying the input data, each connection parameter being represented by a weight coefficient and a bias .theta.,
- wherein the neural network input layer includes means for inputting data;
- the neural network intermediate layer includes means for receiving data from the input layer and processing the received data by means of the connection parameters;
- the neural network output layer includes means for receiving data processed by the intermediate layer, processing the received data by means of the connection parameters and outputting signals indicating one of the predetermined k classes;
- and
- the determining means comprises:
- generating means for generating data representing an initial value .theta..sub.10 of the bias of each connection parameter of the intermediate layer from a total average x of input data x to be input to synapses of the input layer and a weight coefficient W.sub.10 of each connection parameter of the intermediate layer;
- and
- setting means for setting the generated initial values .theta..sub.10 and W.sub.10 into the respective synapses of the intermediate layer, as beginning connection parameters for the back propagation type neural network.
A method for classifying input data representing a physical quantity into predetermined k classes, each class designating a set including a plurality of data which have a similar characteristic., by using a back propagation type neural network having an input layer, an output layer and an intermediate layer coupled between the input and output layers, each layer comprising a network of synapse elements and connections between the synapse elements, each connection being represented by a connection parameter which includes at least a weight coefficient and a bias .theta., the method comprising the steps of:
- (a) accepting plural input data x as vectors;
- (b) generating an average vector x.sub.tot of all vectors x by performing a statistical processing on input data;
- (c) forming an initial value .theta..sub.10 of the intermediate layer from the average vector x.sub.tot and the weight coefficient W.sub.10 of the intermediate layer;
- (d) determining said connection parameters beginning with the initial value .theta..sub.10 according to a back propagation method;
- (e) inputting input data to the input layer;
- (f) processing the input data through the intermediate layer having the determined weight coefficients;
- and
- (g) further processing the data processed through the intermediate layer by means of the output layer, and outputting from the output layer signals indicating one of the predetermined k classes to which the input data belongs.
An apparatus for classifying input data representing a physical quantity into predetermined k classes, each class designating a set including a plurality of data which have similar characteristics, said apparatus comprising:
- a back propagation type neural network including an input layer, an output layer and an intermediate layer coupled between the input layer and the output layer;
- and
- determining means for determining connection parameters between the layers of said neural network by using training data prior to classifying the input data, the connection parameters being represented by at least a weight coefficient,
- wherein, the neural network input layer includes means for inputting data;
- the neural network intermediate layer includes means for receiving data from the input layer and processing the received data by means of the connection parameters;
- the neural network output layer includes means for receiving data processed by the intermediate layer, processing the received data by means of the connection parameters and outputting signals indicating one of the predetermined k classes;
- and
- the determining means comprises generating means for generating data representing a weight coefficient matrix W.sub.20 of the output layer from data representing an average vector x.sup.(k) of each class of input data of the input layer, an initial value W.sub.10 of a weight coefficient matrix of the intermediate layer, and a bias .theta..sub.10 representing an initial value of a bias of the intermediate layer as beginning values for the back propagation type neural network.
A method for classifying input data representing a physical quantity into predetermined k classes, each class designating a set including a plurality of data which have a similar characteristic, by using a back propagation type neural network having an input layer, an output layer and an intermediate layer coupled between the input and output layers, each layer comprising a network of synapse elements and connections between the synapse elements, each connection being represented by a connection parameter which includes at least a weight coefficient and a bias .theta., the method comprising the steps of:
- (a) partitioning input data x to classes as vectors;
- (b) generating an average vector x.sup.(k) for each class of input data of the input layer, an initial value W.sub.10 of a weight coefficient of the intermediate layer, and a vector .theta..sub.10 representing an initial value of a bias of the intermediate layer by performing a statistical process on the input data;
- (c) generating a weight coefficient matrix W.sub.20 of the output layer from the average vector x.sup.(k) the initial value W.sub.10 of a weight coefficient matrix of the intermediate layer, and the initial value vector .theta..sub.10 of the bias of the intermediate layer;
- (d) determining the connection parameters beginning with these initial values according to a back propagation method;
- (e) inputting input data to the input layer;
- (f) processing the input data through the intermediate layer having the determined weight coefficients;
- and
- (g) further processing the data processed through the intermediate layer by means of the output layer, and outputting from the output layer signals indicating one of the predetermined k classes to which the input data belongs.
An apparatus for classifying input data representing a physical quantity into predetermined k classes, each class designating a set including a plurality of data which have similar characteristics, said apparatus comprising:
- a back propagation type neural network including an input layer, an output layer, and an intermediate layer coupled between the input layer and the output layer;
- and
- determining means for determining connection parameters between the layers of said neural network by using training data prior to classifying the input data, the connection parameters being represented at least by a bias,
- wherein, the neural network input layer includes means for inputting data;
- the neural network intermediate layer includes means for receiving data from the input layer and processing the received data by means of the connection parameters;
- the neural network output layer includes means for receiving data processed by the intermediate layer, processing the received data by means of the connection parameters and outputting signals indicating one of the predetermined k classes;
- and
- the determining means comprises means for generating a vector .theta..sub.20 representing an initial value of a bias of the output layer from an initial value W.sub.20 of a weight coefficient matrix of the output layer as a beginning value for the back propagation type neural network.
A method for classifying input data representing a physical quantity into predetermined k classes, each class designating a set including a plurality of data which have a similar characteristic, by using a back propagation type neural network having an input layer, an output layer and an intermediate layer coupled between the input and output layers, each layer comprising a network of synapse elements and connections between the synapse elements, each connection being represented by a connection parameter which includes at least a weight coefficient and a bias .theta., the method comprising the steps of:
- (a) partitioning input data x for classes as vectors;
- (b) generating an initial value W.sub.20 of a weight coefficient of the output layer by performing a statistical process on the input data;
- (c) generating an initial value .theta..sub.20 of a vector representing a bias of the output layer from the initial value W20 of the weight coefficient of the output layer;
- (d) determining the connection parameters beginning with these initial values according to a back propagation method;
- (e) inputting input data to the input layer;
- (f) processing the input data through the intermediate layer having the determined weight coefficients;
- and
- (g) further processing the data processed through the intermediate layer by means of the output layer, and outputting from the output layer signals indicating one of the predetermined k classes to which the input data belongs.
Dependencies are inferred from the claim text.