Methods and systems for training a decision-tree based machine learning algorithm (mla)

A method of training a decision-tree based Machine Learning Algorithm (MLA), the decision-tree based MLA being based on a plurality of generated trees, a given one from the plurality of generated trees having a respective plurality of leaf nodes having respective leaf values,

• the method executable by a server having access to a training dataset, the training dataset comprising a plurality of training objects and a plurality of target values, a given one of the plurality of target values being indicative of a ground-truth associated with a given one of the plurality of training objects,
• the method comprising:
• during a given training iteration of the decision-tree based MLA:
• generating, by the server, a plurality of prediction values using a current plurality of generated trees,
• a given one from the plurality of prediction values being indicative of an output generated by the current plurality of generated trees for the target value of the respective one of the plurality of training objects;
• generating, by the server, a plurality of estimated gradient values by applying a loss function,
• a given one from the plurality of estimated gradient values being indicative of a difference between the target value and the respective one of the plurality of prediction values;
• generating, by the server, a first plurality of noisy estimated gradient values by applying a first noise-injecting function onto the plurality of estimated gradient values;
• generating, by the server, a plurality of noisy candidate trees using the first plurality of noisy estimated gradient values,
• a given one of the plurality of noisy candidate trees having a respective plurality of leaf nodes with respective leaf values, the respective leaf values being based on the first plurality of noisy estimated gradient values,
• applying, by the server, a selection metric to the leaf values of the plurality of noisy candidate trees to select a target tree amongst the plurality of noisy candidate trees,
• the target tree being a given noisy candidate tree having leaf values that are closest to respective noisy estimated gradient values from the first plurality of noisy estimated gradient values than leaf values of any other noisy candidate tree;
• generating, by the server, a second plurality of noisy estimated gradient values by applying a second noise-injecting function onto the plurality of estimated gradient values;
• generating, by the server, an iteration-specific tree by determining new leaf values for respective leaf nodes of the target tree, the new leaf values being determined based on the second plurality of noisy estimated gradient values;
• and
• storing, by the server, the iteration-specific tree to be used, by the server, in combination with the current plurality of generated trees.