Classification of sparsely labeled text documents while preserving semantics

A neural network system executing on a computer system comprising:

• a first plurality of processing elements configured to extract a plurality of local features and a sequence of the local features from a text corpus, wherein the text corpus is sparsely labeled;
• a second plurality of processing elements configured to extract a plurality of global characteristics of a text corpus, wherein the processing elements are structured as capsules configured to preserve the sequence of the local features;
• and
• a third plurality of processing elements configured to extract a plurality of global sequential text dependencies from the text corpus given an output of the first plurality of processing elements, wherein the third plurality of processing elements are structured as a long short-term memory layer, which is configured to output a probability distribution over all labels generated by the first and second plurality of processing elements.

A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for training a neural network to classify sparsely labeled text documents while preserving semantics comprising:

• receiving a text corpus containing a labeled portion and an unlabeled portion exceeding the labeled portion;
• extracting a plurality of local n-gram features and a sequence of the local n-gram features from the text corpus;
• processing the text corpus, using a plurality of convolutional layers, according to the local n-gram features to determine capsule parameters of a plurality of capsules configured to preserve the sequence of the local n-gram features;
• performing a forward-oriented dynamic routing between the plurality of capsules using the capsule parameters to extract a plurality of global characteristics of the text corpus;
• and
• processing the text corpus according to the global characteristics using a long short-term memory layer to extract a plurality of global sequential text dependencies from the text corpus,
• wherein a plurality of parameters of the neural network are updated according to the local n-gram features, the capsule parameters, global characteristics, and global sequential text dependencies.