US 2025/0209385 A1Published application

Covariate drift detection

AssigneeSage Global Services Limited

InventorsYu-Cheng TSAI, Edolfo GARZA-LICUDINE

Life of this patent

Estimating the term…

Loading patent…

Abstract

A computer implemented method for covariate drift correction in a system that employs an AI model trained on a first training dataset for generating output prediction data. A covariate shift quantification process is applied to input data, including computing a statistical value to quantify the drift in the input data. The statistical value is compared with a predetermined threshold to determine if a covariate shift has occurred. A retraining process is triggered for the AI model in response to a covariate shift occurring. A further training dataset is retrieved based on input data and prediction data after the first training dataset was generated. Candidate training datasets are generated from the further training dataset, by applying a different combination of different temporal windows and different temporal weighting decay rates. The candidate training datasets are evaluated and selected, and the AI model is retained with a selected candidate training dataset.

Claims22

Dependent claims

A computer implemented method for covariate drift correction in a system that employs an Al model trained on a first training dataset for generating output prediction data, the method comprising:

a. receiving input data intended for said AI model;
b. applying a covariate shift quantification process to said input data, wherein said covariate shift quantification process comprises computing a statistical value to quantify the drift in said input data relative to said first training dataset;
c. comparing said statistical value with a predetermined threshold;
d. determining that a covariate shift has occurred when said statistical value exceeds said predetermined threshold;
and
e. triggering a retraining process for said AI model in response to said determination that a covariate shift has occurred, wherein the retraining process comprises the steps of:
f. retrieving a further training dataset comprising at least training data based on input data and prediction data from operation of the system after the first training dataset was generated;
g. generating from the further training dataset a plurality of candidate training datasets, each generated by applying to the further training dataset a different combination of different temporal windows and different temporal weighting decay rates, wherein each different temporal window specifies a different time range for data samples of the training dataset used, and each different temporal weighting decay rate applies a different decay rate which progressively reduces the impact of each data sample on the training of the AI model the less recent the data sample;
h. evaluating performance of said plurality of candidate training datasets using model simulations trained on each of said candidate training datasets and tested against a benchmark dataset;
and
i. selecting the candidate training dataset with the combination of temporal window and temporal weighting decay rate that yields the highest performance in said evaluation for retraining said AI model, and
j. retraining the AI model with the selected candidate training dataset.

9 more elements

A computer system for covariate drift correction, said system comprising a covariate drift detection unit and a model retraining module, said covariate drift detection unit configured to:

receive input data intended for an AI model;
apply a covariate shift detection process to said input data, wherein said covariate shift quantification process comprises computing a statistical value to quantify the drift in said input data relative to a first training dataset on which the AI model was trained;
compare said statistical value with a predetermined threshold;
determine that a covariate shift has occurred when said statistical value exceeds said predetermined threshold;
and
trigger the model retraining module in response to said determination that a covariate shift has occurred, wherein, upon triggering, the model retraining module is configured to:
retrieve a further training dataset comprising at least training data based on input data and prediction data from operation of the system after the first training dataset was generated;
generate from the further training dataset a plurality of candidate training datasets, each generated by applying to the further training dataset a different combination of different temporal windows and different temporal weighting decay rates, wherein each different temporal window specifies a different time range for data samples of the training dataset used, and each different temporal weighting decay rate applies a different decay rate which progressively reduces the impact of each data sample on the training of the AI model;
evaluate performance of said plurality of candidate training datasets using model simulations trained on each of said candidate training datasets and tested against a benchmark dataset;
select the candidate training dataset with the combination of temporal window and temporal weighting decay rate that yields the highest performance in said evaluation for retraining said AI model, and
retrain the AI model with the selected candidate training dataset.

8 more elements

Dependencies are inferred from the claim text.