Method and apparatus for extending neural network

A processor-implemented method, the method comprising:

• implementing a neural network;
• selecting a node of the neural network;
• extending the neural network by adding a new node in a layer, of the neural network, that includes the selected node;
• configuring a selected portion of connection weights of the new node, using a selected portion of connection weights of the selected node,
• wherein the configuring comprises:
• dividing the connection weights of the selected node into a first group and a second group so that a difference between a sum of connection weights belonging to the first group and a sum of connection weights belonging to the second group is less than or equal to a predetermined fifth threshold value;
• selecting connection weights belonging to one of the first group and the second group;
• and
• configuring at least the selected portion of the connection weights of the new node, using the connection weights of the selected group;
• and
• implementing the extended neural network with the configured selected portion of the connection weights of the new node.

A processor-implemented method of performing dimension reduction in a neural network, the method comprising:

• training the neural network comprising an input layer, one or more hidden layers, and an output layer;
• determining whether an extension of a hidden layer, among the one or more hidden layers of the neural network, is required during the training of the neural network;
• generating activation frequency information based on determined activation patterns of nodes of the one or more hidden layers of the neural network;
• and
• in response to a determination that the extension of the hidden layer is required, performing the dimension reduction by:
• determining a node of the hidden layer using the generated activation frequency information;
• extending the hidden layer by adding a new node to the hidden layer including the determined node;
• configuring a selected portion of connection weights of the new node, using a selected portion of connection weights of the determined node in the hidden layer;
• and
• configuring at least a remaining portion of the connection weights of the new node, using a predetermined initial value,
• wherein the determining of the node of the hidden layer using the generated activation frequency information comprises determining the determined node based on the generated activation frequency information and generated activation entropy information of the nodes of the neural network.