Neural network training method and apparatus, and data processing apparatus

A processor-implemented neural network training method based on training data, the method comprising:

• receiving training data comprising sequential data to be used in a plurality of time intervals of a training of a recurrent neural network;
• selecting, from a hidden layer in the neural network, an identical reference hidden node to be continuously excluded during each of the plurality of time intervals of the training of the neural network, wherein unselected nodes of the layer include remaining nodes;
• and
• in response to the selecting of the identical reference hidden node, performing the training for each time interval and subsequent time interval among the plurality of time intervals based on the sequential data and by continuously excluding one or more recurrent connections between the selected identical reference hidden node of the layer in the time interval and one or more of hidden nodes included in the remaining nodes of the layer in the subsequent time interval.

A neural network training apparatus based on training data, the apparatus comprising:

• a training data storage configured to store training data comprising sequential data to be used in a plurality of time intervals of a training of a recurrent neural network;
• and
• a processor configured to:
• select, from a layer in the neural network, an identical reference hidden node to be excluded during each of the plurality of time intervals of the training of the neural network, and
• in response to the selecting of the identical reference hidden node, perform the training for each time interval and subsequent time interval among the plurality of time intervals based on the sequential data and by continuously excluding one or more recurrent connections between the selected identical reference hidden node of a layer of the neural network in the time interval and one or more of the hidden nodes of the layer in a subsequent time interval.

A processor-implemented neural network training method comprising:

• receiving a training input at an input layer of a neural network;
• selecting, from a hidden layer of the neural network, a reference hidden node to be continuously ignored during each of a plurality of time intervals of a training of the neural network;
• and
• in response to the selecting of the identical reference hidden node, performing the training for each time interval and subsequent time interval among the plurality of time intervals based the input training data by continuously ignoring one or more connections between the identical reference hidden node of the hidden layer in the time interval and one or more nodes in another hidden layer in the time interval, and by continuously ignoring one or more recurrent connections between the identical reference hidden node of the hidden layer in the time interval and one or more hidden nodes in the hidden layer in a subsequent time interval.