Methods for development of a machine learning system through layered gradient boosting

A method, comprising:

• retrieving a training dataset from at least one database;
• converting a plurality of records in the training dataset to categorical variables in numeric representation, wherein each of the plurality of records includes one or more attribute variables associated with historical customer data and a response variable representing a known insurance premium outcome;
• selecting a first set of numerical parameters and a second set of numerical parameters;
• selecting a loss function based on a probability distribution;
• initializing a model having a plurality of decision trees with different depths, based on the training dataset, the first set of numerical parameters, the second set of numerical parameters, the selected loss function, and a third set of numerical parameters, to compute a plurality of model parameters;
• training the model, based on the training dataset, to refine a plurality of model parameters of the plurality of decision trees through a plurality of iterations, wherein each iteration comprises:
• computing a first-order derivative of the selected loss function and a second-order derivative of the selected loss function based on the training dataset, the first set of numerical parameters, and a first set of model parameters of decision trees in equal or lower depths in prior iterations;
• determining splits of the plurality of decision trees based on comparison results between the second-order derivative of the selected loss function and a first element of the second set of numerical parameters;
• computing a marginal parameter based on the ratio of the computed first-order derivative of the selected loss function and the computed second-order derivative of the selected loss function;
• updating the model parameters of the plurality of decision trees with a product of the marginal parameter and a second element of the second set of numerical parameters based on a second set of model parameters of decision trees in lower depths of all iterations and in the equal depth in prior iterations;
• determining that the trained model, after training through the plurality of iterations, satisfies at least one of stopping criteria;
• and
• storing the splits and the plurality of model parameters of the plurality of decision trees within the trained model.

A non-transitory computer-readable medium including processor-executable instructions for generating a layered machine learning model to proceed data to predict at least one of insurance premium policies, claim cost, claim frequency, and claim severity, when executed by a processor, cause the processor to perform the steps of:

• retrieving a training dataset from at least one database;
• converting a plurality of records in the training dataset to categorical variables in numeric representation, wherein each of the plurality of records includes one or more attribute variables associated with historical customer data and a response variable representing a known insurance premium outcome;
• selecting a plurality of weight variables for the plurality of records in the training dataset and a plurality of hyperparameters including minimum child weight, learning rate, minimum split loss, number of iterations, maximum depth of the decision tree, row sampling, column sampling by tree, and column sampling by split;
• selecting a loss function based on one of Gaussian (normal) distribution, Poisson distribution, gamma distribution, Tweedie distribution, and logistic distribution;
• initializing a model having a plurality of decision trees with different depths, based on the training dataset, the weight variables, the plurality of hyperparameters, the selected loss function, and a plurality of starting values including a cutoff value for a selected attribute and a predicted value in the first iteration, to compute a plurality of model parameters;
• training the model, based on the training dataset, to refine a plurality of model parameters of the plurality of decision trees through a plurality of iterations, wherein each iteration comprises:
• computing a first-order derivative of the selected loss function and a second-order derivative of the selected loss function based on the training dataset, the weight variables, and a first set of model parameters of decision trees in equal or lower depths in prior iterations;
• determining splits of the plurality of decision trees based on comparison results between the second-order derivative of the selected loss function and a minimum child weight;
• computing a marginal parameter based on the ratio of the computed first-order derivative of the selected loss function and the computed second-order derivative of the selected loss function;
• updating the model parameters of the plurality of decision trees with a product of the marginal parameter and a learning rate based on a second set of model parameters of decision trees in lower depths of all iterations and in the equal depth in prior iterations;
• determining that the trained model, after training through the plurality of iterations, satisfies at least one of stopping criteria, including a maximum number of iterations specified in the second set of numerical parameters, a threshold value indicating no additional gain to be found in a new training iteration, and a threshold value of performance evaluation of the model based on a validation set;
• and
• storing the splits and the plurality of model parameters of the plurality of decision trees within the trained model.