Boosting classification and regression tree performance with dimension reduction

A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable by a device to cause the device to:

• receive a training set;
• initialize the decision tree by constructing a root node and training a root solver with the training set;
• grow the decision tree by iteratively splitting nodes of the decision tree, wherein at a node of the decision tree, dimension reduction is performed on features of data of the training set received at the node, and the data having reduced dimension is split based on optimizing a routing function, for routing to another node of the decision tree, wherein the dimension reduction and the split is performed together at the node, wherein the decision tree includes routing nodes and leaf nodes;
• and
• perform training for the routing functions at the routing nodes, solvers at the leaf nodes, and dimension reduction at each of the routing nodes and leaf nodes of the decision tree simultaneously by a unified optimization algorithm that combines the routing functions at the routing nodes, solvers at the leaf nodes, and dimension reduction at each of the routing nodes and leaf nodes of the decision tree, simultaneously training obtaining model parameters for the routing nodes and solvers for the leaf nodes,
• the dimension reduction being performed on a cluster of data sent to each of the routing nodes, wherein a first cluster sent to a first routing node of the routing nodes has a different set of feature from a second cluster sent to a second routing node of the routing nodes,
• wherein, at each layer below the root node of the decision tree, a solver is assigned the trained root solver and at least two new solvers are trained with a random subset of the training set and fitted as the routing functions.

A system comprising:

• a processor;
• and
• a memory device coupled with the processor,
• the processor configured to at least:
• receive a training set;
• initialize the decision tree by constructing a root node and training a root solver with the training set;
• grow the decision tree by iteratively splitting nodes of the decision tree, wherein at a node of the decision tree, dimension reduction is performed on features of data of the training set received at the node, and the data having reduced dimension is split based on optimizing a routing function, for routing to another node of the decision tree, wherein the dimension reduction and the split is performed together at the node, wherein the decision tree includes routing nodes and leaf nodes;
• perform training for routing functions at the routing nodes, solvers at the leaf nodes, and dimension reduction at each of the routing nodes and leaf nodes of the decision tree simultaneously by a unified optimization algorithm that combines the routing functions at the routing nodes, solvers at the leaf nodes, and dimension reduction at each of the routing nodes and leaf nodes of the decision tree, simultaneously training obtaining model parameters for the routing nodes and solvers for the leaf nodes,
• the dimension reduction being performed on a cluster of data sent to each of the routing nodes, wherein a first cluster sent to a first routing node of the routing nodes has a different set of feature from a second cluster sent to a second routing node of the routing nodes,
• wherein, at each layer below the root node of the decision tree, a solver is assigned the trained root solver and at least two new solvers are trained with a random subset of the training set and fitted as the routing functions.