Real drift detector on partial labeled data in data streams

A computerized-method for real-time detection of real concept drift in predictive machine learning models, by processing high-speed streaming data, said computerized-method comprising:

• in a computerized-system comprising a processor and a memory, receiving by the processor, a real-time data stream having labeled and unlabeled instances,
• every preconfigured period of time, performing by the processor:
• (i) obtaining a window of ‘n’ instances from the data stream having a portion of the ‘n’ instances as reliable labels;
• (ii) computing posterior distribution of the reliable labels of the portion of the ‘n’ instances;
• and
• (iii) operating a Drift-Detection (DD) module, said DD module is configured to:
• a. operating a kernel density estimation on the computed posterior distribution for sensitivity control of the DD module;
• b. operating an error rate function on the estimated kernel density to yield an error value;
• and
• c. training an incremental estimator module, according to the kernel density estimation;
• when the error value is not above a preconfigured drift threshold repeat operations (i) through (iii),
• else when the error value is above the preconfigured drift threshold, at least one real concept drift related action takes place.