Localizing tree-based convolutional neural networks

#The present invention relates generally to the field of neural networks, and more particularly to tree-based convolutional neural networks including a designated pivotal node.

#Deep neural networks have been traditionally used for natural language processing (NLP) applications. Deep neural networks have multiple layers where each layer of the neural network trains on a set of features based on the output of the previous layer. This hierarchical approach allows deep neural networks to model complex non-linear relationships. Deep neural networks include many different approaches depending on the type of data being trained. Some types of deep neural networks include recurrent neural networks (RNNs) and convolutional deep neural networks (CNNs), among others.

#Embodiments of the present invention disclose a method, a computer program product, and a system for localizing tree-based convolutional neural networks. The method may include one or more processors creating a first tree-based convolution layer (TBCL) corresponding to a tree. The tree may include a first plurality of nodes and a node that has been indicated to be a first pivotal node. The first TBCL may include a second plurality of nodes and wherein the first TBCL includes a second pivotal node having a feature vector based on node data from the first pivotal node. The method may further include one or more processors creating a second TBCL corresponding to the tree. The second TBCL may include a third plurality of nodes. The method may further include one or more processors determining a feature vector for the third pivotal node in the third plurality of nodes using the feature vectors from: (i) the second pivotal node, (ii) a parent node of the second pivotal node, and (iii) a child node of the second pivotal node.

#FIG. 1 is a functional block diagram illustrating a distributed data processing environment, in accordance with an embodiment of the present invention;

#FIG. 2 is a flowchart depicting operational steps of a TBCNN program for analyzing subtrees in a TBCNN, in accordance with an embodiment of the present invention;

#FIG. 3 is an illustration depicting a computer program, in accordance with an embodiment of the present invention;

#Tree-based convolutional neural networks (TBCNN) are a novel approach to deep neural networks. Tree-based analysis in TBCNNs is achieved by creating one or more tree-based convolutional (TBC) layers including a tree where each node includes a feature vector based on an input tree. Each node is transformed to a node in the next TBC layer with features of the node and its descendants as well as weight and bias parameters. The last TBC layer is connected to a conventional neural network (e.g., a fully connected layer) by extracting feature vectors via a dynamic pooling.

#TBCNNs allow efficient analysis of tree data structures using deep neural networks. Analysis of tree data structures is useful in natural language processing and other applications. For example, TBCNNs allow deep neural networks to analyze computer programs represented as abstract syntax trees (ASTs). TBCNNs have also shown promise in other areas where tree-based analysis is necessary. However, a shortcoming of TBCNN is that it is difficult to analyze subtrees and it is not possible to specify nodes to extract feature vectors. Additionally, it is unclear whether extracting maximum values among all nodes is always good for subtree analysis. These problems are more acute when trees with high number of nodes are given.

#The present invention analyzes subtrees in a TBCNN by determining a pivotal node and gathering feature vectors from all nodes connected to the pivotal node. Feature vectors are gathered from all nodes connected to the pivotal node. Implementation of embodiments of the present invention may take a variety of forms, and exemplary implementation details are discussed subsequently with reference to the Figures.