# US 2022/0277232 A1 · Computer-based systems, computing components and computing objects configured to implement dynamic outlier bias reduction in machine learning models > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2022/0277232 A1 - Document ID: US20220277232A1 - Status: Published application - Kind: A1 - Application number: 17/572,566 - Filed: 2022-01-10 - Published: 2022-09-01 - Earliest claimed: 2019-09-18 - Assignees: Hartford Steam Boiler Inspection and Insurance Company - Applicants: Hartford Steam Boiler Inspection and Insurance Company - Inventors: RICHARD B. JONES - Web page: https://patentgrep.com/patent/US20220277232A1 ## CPC - G06N 20/00: Machine learning - G06N 20/20: Ensemble learning - G06N 3/08: Learning methods - G06N 3/09: Supervised learning - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Systems and methods include processors for receiving training data for a user activity; receiving bias criteria; determining a set of model parameters for a machine learning model including: (1) applying the machine learning model to the training data; (2) generating model prediction errors; (3) generating a data selection vector to identify non-outlier target variables based on the model prediction errors; (4) utilizing the data selection vector to generate a non-outlier data set; (5) determining updated model parameters based on the non-outlier data set; and (6) repeating steps (1)-(5) until a censoring performance termination criterion is satisfied; training classifier model parameters for an outlier classifier machine learning model; applying the outlier classifier machine learning model to activity-related data to determine non-outlier activity-related data; and applying the machine learning model to the non-outlier activity-related data to predict future activity-related attributes for the user activity. ## Claims ### Claim 1 A method comprising: receiving, by at least one processor, a training data set of target variables representing at least one activity-related attribute for at least one user activity; determining, by the at least one processor, a data selection vector to select non-outlier data elements of the training data set comprising: (1) applying, by the at least one processor, a reference machine learning model to the training data set to determine a set of model predicted values, wherein the reference machine learning model comprises a set of model parameters; (2) generating, by the at least one processor, an error set of data element errors by comparing the set of model predicted values to corresponding actual values of the training data set; (3) generating, by the at least one processor, the data selection vector to identify non-outlier target variables based at least in part on the error set of data element errors and at least one bias criteria; (4) utilizing, by the at least one processor, the data selection vector on the training data set to generate a non-outlier data set; (5) determining, by the at least one processor, a set of updated model parameters for the reference machine learning model based on the non-outlier data set; and (6) repeating, by the at least one processor, steps (1)-(5) as an iteration until at least one censoring performance termination criterion is satisfied; generating, by the at least one processor, a final non-outlier dataset based at least in part on the data selection vector; training, by the at least one processor, based at least in part on the final non-outlier data set, a set of base model parameters of a base machine learning model to obtain a trained base machine learning model trained to predict values of non-outlier data elements; and outputting, by the at least one processor, the trained base machine learning model. ### Claim 2 (depends on 1) The method of claim 1, further comprising: training, by the at least one processor, based at least in part on the final non-outlier data set, a set of user engagement prediction model parameters of a user engagement prediction machine learning model to obtain a trained user engagement prediction machine learning model trained to predict at least one user engagement metric associated with at least one content item of at least one webpage; and wherein the training data set comprises the target variables of historical user engagement with at least one historical content item of at least one historical webpage. ### Claim 3 (depends on 1) The method of claim 1, further comprising: training, by the at least one processor, based at least in part on the final non-outlier data set, a set of grid energy prediction model parameters of a grid energy prediction machine learning model to obtain a trained grid energy prediction machine learning model trained to predict values of future grid energy utilization; and wherein the training data set comprises the target variables of historical grid energy utilization. ### Claim 4 (depends on 1) The method of claim 1, further comprising: applying, by the at least one processor, the data selection vector to the training data set to determine an outlier training data set; training, by the at least one processor, using the outlier training data set, at least one outlier-specific model parameter of at least one outlier-specific machine learning model to predict outlier data values; and outputting, by the at least one processor, the at least one outlier-specific machine learning model. ### Claim 5 (depends on 1) The method of claim 1, further comprising: generating, by the at least one processor, an outlier data set based at least in part on the data selection vector; training, by the at least one processor, based at least in part on the outlier data set, a set of outlier classifier model parameters of an outlier classifier machine learning model to obtain a trained outlier classifier machine learning model trained to classify outlier data elements; and outputting, by the at least one processor, the trained outlier classifier machine learning model. ### Claim 6 (depends on 1) The method of claim 1, further comprising: training, by the at least one processor, using the training data set, generalized model parameters of a generalized machine learning model to predict data values; utilizing, by the at least one processor, the generalized machine learning model to predict outlier activity-related data values for the set of outlier activity-related data; and utilizing, by the at least one processor, the generalized machine learning model to predict the activity-related data values. ### Claim 7 (depends on 1) The method of claim 1, further comprising: applying, by the at least one processor, the data selection vector to the training data set to determine an outlier training data set; training, by the at least one processor, using the outlier training data set, an outlier-specific model parameters of an outlier-specific machine learning model to predict outlier data values; training, by the at least one processor, using the training data set, generalized model parameters of a generalized machine learning model to predict data values; utilizing, by the at least one processor, the outlier-specific machine learning model to predict outlier activity-related data values for the set of outlier activity-related data; and utilizing, by the at least one processor, the outlier-specific machine learning model to predict the activity-related data values. ### Claim 8 (depends on 1) The method of claim 1, further comprising: generating, by the at least one processor, an outlier data set and a non-outlier dataset based at least in part on the data selection vector; training, by the at least one processor, using the non-outlier dataset, generalized model parameters of a generalized machine learning model to predict data values; training, by the at least one processor, based at least in part on the outlier data set, a set of outlier classifier model parameters of an outlier classifier machine learning model to obtain a trained outlier classifier machine learning model trained to classify outlier data elements; utilizing, by the at least one processor, the generalized machine learning model to predict the activity-related data values for the set of activity-related data; utilizing, by the at least one processor, the outlier classifier machine learning model to identify outlier activity-related data values of the activity-related data values; and removing, by the at least one processor, the outlier activity-related data values. ### Claim 9 (depends on 1) The method of claim 1, further comprising: splitting, by the at least one processor, the set of activity-related data into a plurality of subsets of activity-related data; determining, by the at least one processor, an ensemble model for each subset of activity-related data of the plurality of subsets of activity-related data; wherein the machine learning model comprises an ensemble of models; wherein each ensemble model comprises a random combination of models from the ensemble of models; utilizing, by the at least one processor, each ensemble model separately to predict ensemble-specific activity-related data values; determining, by the at least one processor, an error for each ensemble model based on the ensemble-specific activity-related data values and known values; and selecting, by the at least one processor, a highest performing ensemble model based on a lowest error. ### Claim 10 A method comprising: transmitting, by at least one processor, a bias reduced model generation request to bias reduced model generation service; wherein the bias reduced model generation request comprises a training data set of target variables representing at least one activity-related attribute for at least one user activity; wherein the bias reduced model generation service is configured to use the training data set to performing steps to: determine a data selection vector to select non-outlier data elements of the training data set comprising: (1) apply a reference machine learning model to the training data set to determine a set of model predicted values, wherein the reference machine learning model comprises a set of model parameters; (2) generate an error set of data element errors by comparing the set of model predicted values to corresponding actual values of the training data set; (3) generate the data selection vector to identify non-outlier target variables based at least in part on the error set of data element errors and at least one bias criteria; (4) utilize the data selection vector on the training data set to generate a non-outlier data set; (5) determine a set of updated model parameters for the reference machine learning model based on the non-outlier data set; and (6) repeat steps (1)-(5) as an iteration until at least one censoring performance termination criterion is satisfied; generate an outlier data set and a non-outlier dataset based at least in part on the data selection vector; train, based at least in part on the non-outlier data set, a set of non-outlier model parameters of a base machine learning model to obtain a trained base machine learning model trained to predict values of non-outlier data elements; and receiving, by the at least one processor from the bias reduced model generation service, the trained base machine learning model. ### Claim 11 (depends on 10) The method of claim 10, further comprising: utilizing, by the at least one processor, the trained base machine learning model to predict the activity-related data values. ### Claim 12 A system comprising: at least one processor in communication with a non-transitory computer-readable storage medium having software instructions stored thereon, wherein the software instructions, when executed, cause the at least one processor to perform steps to: receive a train data set of target variables representing at least one activity-related attribute for at least one user activity; determine data selection vector to select non-outlier data elements of the train data set comprising: (1) applying a reference machine learning model to the train data set to determine a set of model predicted values, wherein the reference machine learning model comprises a set of model parameters; (2) generating an error set of data element errors by comparing the set of model predicted values to corresponding actual values of the train data set; (3) generating the data selection vector to identify non-outlier target variables based at least in part on the error set of data element errors and at least one bias criteria; (4) utilizing the data selection vector on the train data set to generate a non-outlier data set; (5) determining a set of updated model parameters for the reference machine learning model based on the non-outlier data set; and (6) repeating steps (1)-(5) as an iteration until at least one censoring performance termination criterion is satisfied; generate a final non-outlier dataset based at least in part on the data selection vector; train, based at least in part on the final non-outlier data set, a set of base model parameters of a base machine learning model to obtain a trained base machine learning model trained to predict values of non-outlier data elements; and output the trained base machine learning model. ### Claim 13 (depends on 12) The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform steps to: train based at least in part on the final non-outlier data set, a set of user engagement prediction model parameters of a user engagement prediction machine learning model to obtain a trained user engagement prediction machine learning model trained to predict at least one user engagement metric associated with at least one content item of at least one webpage; and wherein the train data set comprises the target variables of historical user engagement with at least one historical content item of at least one historical webpage. ### Claim 14 (depends on 12) The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform steps to: train based at least in part on the final non-outlier data set, a set of grid energy prediction model parameters of a grid energy prediction machine learning model to obtain a trained grid energy prediction machine learning model trained to predict values of future grid energy utilization; and wherein the train data set comprises the target variables of historical grid energy utilization. ### Claim 15 (depends on 12) The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform steps to: apply the data selection vector to the train data set to determine an outlier train data set; train using the outlier train data set, at least one outlier-specific model parameter of at least one outlier-specific machine learning model to predict outlier data values; and output the at least one outlier-specific machine learning model. ### Claim 16 (depends on 12) The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform steps to: generate an outlier data set based at least in part on the data selection vector; train based at least in part on the outlier data set, a set of outlier classifier model parameters of an outlier classifier machine learning model to obtain a trained outlier classifier machine learning model trained to classify outlier data elements; output the trained outlier classifier machine learning model. ### Claim 17 (depends on 12) The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform steps to: train using the train data set, generalized model parameters of a generalized machine learning model to predict data values; utilize the generalized machine learning model to predict outlier activity-related data values for the set of outlier activity-related data; and utilize the generalized machine learning model to predict the activity-related data values. ### Claim 18 (depends on 12) The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform steps to: apply the data selection vector to the train data set to determine an outlier train data set; train using the outlier train data set, an outlier-specific model parameters of an outlier-specific machine learning model to predict outlier data values; train using the train data set, generalized model parameters of a generalized machine learning model to predict data values; utilize the outlier-specific machine learning model to predict outlier activity-related data values for the set of outlier activity-related data; and utilize the outlier-specific machine learning model to predict the activity-related data values. ### Claim 19 (depends on 12) The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform steps to: generate an outlier data set and a non-outlier dataset based at least in part on the data selection vector; train, using the non-outlier dataset, generalized model parameters of a generalized machine learning model to predict data values; train, based at least in part on the outlier data set, a set of outlier classifier model parameters of an outlier classifier machine learning model to obtain a trained outlier classifier machine learning model trained to classify outlier data elements; utilize the generalized machine learning model to predict the activity-related data values for the set of activity-related data; utilize the outlier classifier machine learning model to identify outlier activity-related data values of the activity-related data values; and remove the outlier activity-related data values. ### Claim 20 (depends on 12) The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform steps to: split the set of activity-related data into a plurality of subsets of activity-related data; determine an ensemble model for each subset of activity-related data of the plurality of subsets of activity-related data; wherein the machine learning model comprises an ensemble of models; wherein each ensemble model comprises a random combination of models from the ensemble of models; utilize each ensemble model separately to predict ensemble-specific activity-related data values; determine an error for each ensemble model based on the ensemble-specific activity-related data values and known values; and select a highest performing ensemble model based on a lowest error.