Apparatus and network construction method for determining the number of elements in an intermediate layer of a neural network
AssigneeMitsubishi Electric Corporation
InventorsToshisada Mariyama, Kunihiko Fukushima, Wataru Matsumoto
Life of this patent
Estimating the term…
Abstract
An element construction unit compares output values of one or more elements included in an intermediate layer calculated by an output value calculating unit with a threshold value, and the number of elements included in the intermediate layer is maintained when any of the output values out of the output values of the one or more elements included in the intermediate layer is greater than the threshold value, and the number of elements included in the intermediate layer is increased when all of the output values of the one or more elements included in the intermediate layer are equal to or less than the threshold value.
Claims18
A network construction apparatus for constructing a neural network having an input layer, one or more intermediate layers having a number of respective elements, and an output layer, where the input/output response of the respective elements of one or more intermediate layers is determined by a Gaussian function, comprising:
- a processor;
- and
- a memory storing instructions, upon executed by the processor, causing the processor to
- perform an element construction process to determine a structure of the one or more intermediate layers, the element construction processing including processes to:
- determine whether the one or more intermediate layers include at least one element and adding an element to the one or more intermediate layers in response to a determination that non elements currently exist;
- initialize, for each element included in an intermediate layer, parameters of a Gaussian function related to the element when the neural network is constructed, the neural network including the intermediate layer including an element whose input/output response is determined by the Gaussian function;
- calculate output values of one or more elements included in the intermediate layer in accordance with the initialized parameters and an output value of an element included in the input layer of the neural network;
- compare the calculated output values of the one or more elements with a threshold value, and maintaining a number of elements included in the intermediate layer when an output value of any of the elements out of the output values of the one or more elements included in the intermediate layer is greater than the threshold value, and performing element construction processing of increasing the number of elements included in the intermediate layer when all of the output values of the one or more elements included in the intermediate layer are equal to or less than the threshold value;
- and
- repeat the element construction processes until all pieces of learning data have been acquired;
- and
- perform a training process of updating the parameters of the Gaussian function and updating a weight between an element included in the one or more intermediate layers and an element included in the output layer, by performing supervised learning to learn the parameters of the Gaussian function when learning data is given to the input layer after the element construction processing is completed.
A network construction method for constructing a neural network having an input layer, one or more intermediate layers having a number of respective elements, and an output layer, where the input/output response of the respective elements of one or more intermediate layers is determined by a Gaussian function, the method comprising:
- performing an element construction process to determine a structure of the one or more intermediate layers, the element construction processing including:
- determining whether the one or more intermediate layers include at least one element and adding an element to the one or more intermediate layers in response to a determination that non elements currently exist;
- initializing for each element included in an intermediate layer, parameters of a Gaussian function related to the element when the neural network is constructed, the neural network including the intermediate layer including an element whose input/output response is determined by the Gaussian function;
- calculating output values of one or more elements included in the intermediate layer in accordance with the initialized parameters and an output value of an element included in the input layer of the neural network;
- comparing the calculated output values of the one or more elements with a threshold value, and maintaining a number of elements included in the intermediate layer when an output value of any of the elements out of the output values of the one or more elements included in the intermediate layer is greater than the threshold value, and performing element construction processing of increasing the number of elements included in the intermediate layer when all of the output values of the one or more elements included in the intermediate layer are equal to or less than the threshold value, by an element construction step, and
- repeating the element construction processes until all pieces of learning data have been acquired;
- and
- performing a training process of updating the parameters of the Gaussian function and updating a weight between an element included in the one or more intermediate layers and an element included in the output layer, by performing supervised learning to learn the parameters of the Gaussian function when learning data is given to the input layer after the element construction processing is completed.
Dependencies are inferred from the claim text.