Method and apparatus for generating hidden state in recurrent neural network for language processing

An apparatus for generating a hidden state in a recurrent neural network for language processing, the apparatus comprising a memory and a processor, the memory storing a computer program, the computer program, when executed by the processor, causing the processor to:

• generate regional word vectors of at least two dimensions of a target word vector inputted at a first moment based on at least two first weight matrices and a second hidden state corresponding to a second word vector inputted at a second moment being earlier than the first moment in time, each of the first weight matrices being used for generating a corresponding regional word vector;
• combine the regional word vectors in a preset regional vector combination manner to obtain combined regional word vectors of the at least two dimensions, wherein the preset regional vector combination manner includes a graph convolution network combination manner for processing the regional word vectors as vector nodes in a graph convolution network or a capsule network combination manner for processing the regional word vectors as capsules in a capsule network;
• perform aggregation transformation processing on the combined regional word vectors based on a feedforward neural network, to obtain an aggregated word vector corresponding to the target word vector;
• and
• generate a target hidden state corresponding to the target word vector based on the aggregated word vector.

A non-transitory computer-readable storage medium, storing a computer program, the computer program, when executed by a processor, causing the processor to perform:

• generating regional word vectors of at least two dimensions of a target word vector inputted at a first moment based on at least two first weight matrices and a second hidden state corresponding to a second word vector inputted at a second moment being earlier than the first moment in time, each of the first weight matrices being used for generating a corresponding regional word vector;
• combining the regional word vectors in a preset regional vector combination manner to obtain combined regional word vectors of the at least two dimensions, wherein the preset regional vector combination manner includes a graph convolution network combination manner for processing the regional word vectors as vector nodes in a graph convolution network or a capsule network combination manner for processing the regional word vectors as capsules in a capsule network;
• performing aggregation transformation processing on the combined regional word vectors based on a feedforward neural network, to obtain an aggregated word vector corresponding to the target word vector, and
• generating a target hidden state corresponding to the target word vector based on the aggregated word vector.